* [PATCH 00/13] add support for idpf PMD in DPDK
@ 2022-08-03 11:30 Junfeng Guo
2022-08-03 11:30 ` [PATCH 01/13] net/idpf/base: introduce base code Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD [*EXPERIMENTAL*] in DPDK for Intel Device ID of 0x1452.
Junfeng Guo (13):
net/idpf/base: introduce base code
net/idpf/base: add logs and OS specific implementation
net/idpf: support device initialization
net/idpf: add queue operations
net/idpf: add support to get device information
net/idpf: add support to get packet type
net/idpf: add support to update link status
net/idpf: add basic Rx/Tx datapath
net/idpf: add support for RSS
net/idpf: add mtu configuration
net/idpf: add hw statistics
net/idpf: support write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
drivers/net/idpf/base/iecm_alloc.h | 22 +
drivers/net/idpf/base/iecm_common.c | 359 +++
drivers/net/idpf/base/iecm_controlq.c | 662 ++++
drivers/net/idpf/base/iecm_controlq.h | 214 ++
drivers/net/idpf/base/iecm_controlq_api.h | 227 ++
drivers/net/idpf/base/iecm_controlq_setup.c | 179 ++
drivers/net/idpf/base/iecm_devids.h | 17 +
drivers/net/idpf/base/iecm_lan_pf_regs.h | 134 +
drivers/net/idpf/base/iecm_lan_txrx.h | 428 +++
drivers/net/idpf/base/iecm_lan_vf_regs.h | 114 +
drivers/net/idpf/base/iecm_osdep.h | 365 +++
drivers/net/idpf/base/iecm_prototype.h | 45 +
drivers/net/idpf/base/iecm_type.h | 106 +
drivers/net/idpf/base/meson.build | 27 +
drivers/net/idpf/base/siov_regs.h | 41 +
drivers/net/idpf/base/virtchnl.h | 2743 +++++++++++++++++
drivers/net/idpf/base/virtchnl2.h | 1411 +++++++++
drivers/net/idpf/base/virtchnl2_lan_desc.h | 603 ++++
drivers/net/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/net/idpf/idpf_ethdev.c | 1194 +++++++
drivers/net/idpf/idpf_ethdev.h | 247 ++
drivers/net/idpf/idpf_logs.h | 38 +
drivers/net/idpf/idpf_rxtx.c | 2348 ++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 228 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 917 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 85 +
drivers/net/idpf/idpf_vchnl.c | 1206 ++++++++
drivers/net/idpf/meson.build | 47 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
30 files changed, 14578 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_alloc.h
create mode 100644 drivers/net/idpf/base/iecm_common.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_api.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_setup.c
create mode 100644 drivers/net/idpf/base/iecm_devids.h
create mode 100644 drivers/net/idpf/base/iecm_lan_pf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_lan_txrx.h
create mode 100644 drivers/net/idpf/base/iecm_lan_vf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_osdep.h
create mode 100644 drivers/net/idpf/base/iecm_prototype.h
create mode 100644 drivers/net/idpf/base/iecm_type.h
create mode 100644 drivers/net/idpf/base/meson.build
create mode 100644 drivers/net/idpf/base/siov_regs.h
create mode 100644 drivers/net/idpf/base/virtchnl.h
create mode 100644 drivers/net/idpf/base/virtchnl2.h
create mode 100644 drivers/net/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/net/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 01/13] net/idpf/base: introduce base code
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:30 ` [PATCH 02/13] net/idpf/base: add logs and OS specific implementation Junfeng Guo
` (12 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Introduce base code for IDPF (Infrastructure Data Path Function) PMD.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/base/iecm_alloc.h | 22 +
drivers/net/idpf/base/iecm_common.c | 359 +++
drivers/net/idpf/base/iecm_controlq.c | 662 ++++
drivers/net/idpf/base/iecm_controlq.h | 214 ++
drivers/net/idpf/base/iecm_controlq_api.h | 227 ++
drivers/net/idpf/base/iecm_controlq_setup.c | 179 ++
drivers/net/idpf/base/iecm_devids.h | 17 +
drivers/net/idpf/base/iecm_lan_pf_regs.h | 134 +
drivers/net/idpf/base/iecm_lan_txrx.h | 428 +++
drivers/net/idpf/base/iecm_lan_vf_regs.h | 114 +
drivers/net/idpf/base/iecm_prototype.h | 45 +
drivers/net/idpf/base/iecm_type.h | 106 +
drivers/net/idpf/base/meson.build | 27 +
drivers/net/idpf/base/siov_regs.h | 41 +
drivers/net/idpf/base/virtchnl.h | 2743 +++++++++++++++++
drivers/net/idpf/base/virtchnl2.h | 1411 +++++++++
drivers/net/idpf/base/virtchnl2_lan_desc.h | 603 ++++
drivers/net/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
18 files changed, 7899 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_alloc.h
create mode 100644 drivers/net/idpf/base/iecm_common.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_api.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_setup.c
create mode 100644 drivers/net/idpf/base/iecm_devids.h
create mode 100644 drivers/net/idpf/base/iecm_lan_pf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_lan_txrx.h
create mode 100644 drivers/net/idpf/base/iecm_lan_vf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_prototype.h
create mode 100644 drivers/net/idpf/base/iecm_type.h
create mode 100644 drivers/net/idpf/base/meson.build
create mode 100644 drivers/net/idpf/base/siov_regs.h
create mode 100644 drivers/net/idpf/base/virtchnl.h
create mode 100644 drivers/net/idpf/base/virtchnl2.h
create mode 100644 drivers/net/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/net/idpf/base/virtchnl_inline_ipsec.h
diff --git a/drivers/net/idpf/base/iecm_alloc.h b/drivers/net/idpf/base/iecm_alloc.h
new file mode 100644
index 0000000000..7ea219c784
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_ALLOC_H_
+#define _IECM_ALLOC_H_
+
+/* Memory types */
+enum iecm_memset_type {
+ IECM_NONDMA_MEM = 0,
+ IECM_DMA_MEM
+};
+
+/* Memcpy types */
+enum iecm_memcpy_type {
+ IECM_NONDMA_TO_NONDMA = 0,
+ IECM_NONDMA_TO_DMA,
+ IECM_DMA_TO_DMA,
+ IECM_DMA_TO_NONDMA
+};
+
+#endif /* _IECM_ALLOC_H_ */
diff --git a/drivers/net/idpf/base/iecm_common.c b/drivers/net/idpf/base/iecm_common.c
new file mode 100644
index 0000000000..418fd99298
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_common.c
@@ -0,0 +1,359 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "iecm_type.h"
+#include "iecm_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * iecm_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int iecm_set_mac_type(struct iecm_hw *hw)
+{
+ int status = IECM_SUCCESS;
+
+ DEBUGFUNC("iecm_set_mac_type\n");
+
+ if (hw->vendor_id == IECM_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IECM_DEV_ID_PF:
+ hw->mac.type = IECM_MAC_PF;
+ break;
+ default:
+ hw->mac.type = IECM_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IECM_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("iecm_set_mac_type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * iecm_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int iecm_init_hw(struct iecm_hw *hw, struct iecm_ctlq_size ctlq_size)
+{
+ struct iecm_ctlq_create_info *q_info;
+ int status = IECM_SUCCESS;
+ struct iecm_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct iecm_ctlq_create_info *)
+ iecm_calloc(hw, 2, sizeof(struct iecm_ctlq_create_info));
+ if (!q_info)
+ return IECM_ERR_NO_MEMORY;
+
+ q_info[0].type = IECM_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IECM_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IECM_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IECM_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = iecm_ctlq_init(hw, 2, q_info);
+ if (status != IECM_SUCCESS) {
+ /* TODO return error */
+ iecm_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, iecm_ctlq_info, cq_list) {
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. iecm_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int iecm_send_msg_to_cp(struct iecm_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct iecm_ctlq_msg ctlq_msg = { 0 };
+ struct iecm_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = iecm_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct iecm_dma_mem *)
+ iecm_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IECM_ERR_NO_MEMORY;
+
+ iecm_memcpy(dma_mem.va, msg, msglen, IECM_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = iecm_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ iecm_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * iecm_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool iecm_asq_done(struct iecm_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * iecm_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool iecm_check_asq_alive(struct iecm_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * iecm_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int iecm_clean_arq_element(struct iecm_hw *hw,
+ struct iecm_arq_event_info *e, u16 *pending)
+{
+ struct iecm_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = iecm_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ iecm_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IECM_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * iecm_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int iecm_deinit_hw(struct iecm_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return iecm_ctlq_deinit(hw);
+}
+
+/**
+ * iecm_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int iecm_reset(struct iecm_hw *hw)
+{
+ return iecm_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IECM_SUCCESS, NULL, 0);
+}
+
+/**
+ * iecm_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int iecm_get_set_rss_lut(struct iecm_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int iecm_get_rss_lut(struct iecm_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return iecm_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * iecm_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int iecm_set_rss_lut(struct iecm_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return iecm_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * iecm_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int iecm_get_set_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int iecm_get_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key)
+{
+ return iecm_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * iecm_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int iecm_set_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key)
+{
+ return iecm_get_set_rss_key(hw, vsi_id, key, true);
+}
diff --git a/drivers/net/idpf/base/iecm_controlq.c b/drivers/net/idpf/base/iecm_controlq.c
new file mode 100644
index 0000000000..3a877bbf74
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq.c
@@ -0,0 +1,662 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "iecm_controlq.h"
+
+/**
+ * iecm_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+iecm_ctlq_setup_regs(struct iecm_ctlq_info *cq,
+ struct iecm_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * iecm_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void iecm_ctlq_init_regs(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IECM_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IECM_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * iecm_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void iecm_ctlq_init_rxq_bufs(struct iecm_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct iecm_ctlq_desc *desc = IECM_CTLQ_DESC(cq, i);
+ struct iecm_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IECM_CTLQ_FLAG_BUF | IECM_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * iecm_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void iecm_ctlq_shutdown(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ iecm_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ iecm_release_lock(&cq->cq_lock);
+ iecm_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * iecm_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: iecm_ctlq_init must be called prior to any calls to iecm_ctlq_add
+ */
+int iecm_ctlq_add(struct iecm_hw *hw,
+ struct iecm_ctlq_create_info *qinfo,
+ struct iecm_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IECM_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IECM_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IECM_CTLQ_MAX_BUF_LEN)
+ return IECM_ERR_CFG;
+
+ *cq_out = (struct iecm_ctlq_info *)
+ iecm_calloc(hw, 1, sizeof(struct iecm_ctlq_info));
+ if (!(*cq_out))
+ return IECM_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IECM_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+ fallthrough;
+ case IECM_CTLQ_TYPE_MAILBOX_TX:
+ status = iecm_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IECM_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ iecm_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct iecm_ctlq_msg **)
+ iecm_calloc(hw, qinfo->len,
+ sizeof(struct iecm_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IECM_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ iecm_ctlq_setup_regs(*cq_out, qinfo);
+
+ iecm_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ iecm_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ iecm_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void iecm_ctlq_remove(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ iecm_ctlq_shutdown(hw, cq);
+ iecm_free(hw, cq);
+}
+
+/**
+ * iecm_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int iecm_ctlq_init(struct iecm_hw *hw, u8 num_q,
+ struct iecm_ctlq_create_info *q_info)
+{
+ struct iecm_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IECM_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct iecm_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = iecm_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ iecm_ctlq_info, cq_list)
+ iecm_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * iecm_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int iecm_ctlq_deinit(struct iecm_hw *hw)
+{
+ struct iecm_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IECM_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ iecm_ctlq_info, cq_list)
+ iecm_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * iecm_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int iecm_ctlq_send(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ u16 num_q_msg, struct iecm_ctlq_msg q_msg[])
+{
+ struct iecm_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IECM_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IECM_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IECM_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct iecm_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IECM_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IECM_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IECM_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IECM_HOST_ID_MASK) <<
+ IECM_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct iecm_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IECM_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IECM_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(buff->pa));
+
+ iecm_memcpy(&desc->params, msg->ctx.indirect.context,
+ IECM_INDIRECT_CTX_SIZE, IECM_NONDMA_TO_DMA);
+ } else {
+ iecm_memcpy(&desc->params, msg->ctx.direct,
+ IECM_DIRECT_CTX_SIZE, IECM_NONDMA_TO_DMA);
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ iecm_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ iecm_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int iecm_ctlq_clean_sq(struct iecm_ctlq_info *cq, u16 *clean_count,
+ struct iecm_ctlq_msg *msg_status[])
+{
+ struct iecm_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IECM_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IECM_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IECM_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IECM_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ iecm_memset(desc, 0, sizeof(*desc), IECM_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ iecm_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * iecm_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int iecm_ctlq_post_rx_buffs(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ u16 *buff_count, struct iecm_dma_mem **buffs)
+{
+ struct iecm_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IECM_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IECM_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IECM_CTLQ_FLAG_BUF | IECM_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ iecm_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int iecm_ctlq_recv(struct iecm_ctlq_info *cq, u16 *num_q_msg,
+ struct iecm_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct iecm_ctlq_desc *desc;
+ int ret_code = IECM_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IECM_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ iecm_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IECM_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IECM_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IECM_CTLQ_FLAG_FTYPE_VM |
+ IECM_CTLQ_FLAG_FTYPE_PF)) >>
+ IECM_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IECM_CTLQ_FLAG_ERR)
+ ret_code = IECM_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ iecm_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IECM_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ iecm_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IECM_INDIRECT_CTX_SIZE,
+ IECM_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ iecm_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IECM_DIRECT_CTX_SIZE,
+ IECM_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ iecm_memset(desc, 0, sizeof(struct iecm_ctlq_desc),
+ IECM_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ iecm_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IECM_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/net/idpf/base/iecm_controlq.h b/drivers/net/idpf/base/iecm_controlq.h
new file mode 100644
index 0000000000..0964146b49
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq.h
@@ -0,0 +1,214 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_CONTROLQ_H_
+#define _IECM_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "iecm_osdep.h"
+#include "iecm_alloc.h"
+/* This is used to explicitly annotate when a switch case falls through to the
+ * next case.
+ */
+#define fallthrough do {} while (0)
+#endif
+#include "iecm_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IECM_CTLQ_MAX_RING_SIZE 1024
+#define IECM_CTLQ_MAX_BUF_LEN 4096
+
+#define IECM_CTLQ_DESC(R, i) \
+ (&(((struct iecm_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IECM_CTLQ_DESC_UNUSED(R) \
+ (u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1)
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IECM_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IECM_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IECM_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IECM_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IECM_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct iecm_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IECM_CTLQ_DESC_VF_ID_S 0
+#define IECM_CTLQ_DESC_VF_ID_M (0x7FF << IECM_CTLQ_DESC_VF_ID_S)
+#define IECM_CTLQ_DESC_PF_ID_S 11
+#define IECM_CTLQ_DESC_PF_ID_M (0x1F << IECM_CTLQ_DESC_PF_ID_S)
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IECM_CTLQ_FLAG_DD_S 0
+#define IECM_CTLQ_FLAG_CMP_S 1
+#define IECM_CTLQ_FLAG_ERR_S 2
+#define IECM_CTLQ_FLAG_FTYPE_S 6
+#define IECM_CTLQ_FLAG_RD_S 10
+#define IECM_CTLQ_FLAG_VFC_S 11
+#define IECM_CTLQ_FLAG_BUF_S 12
+#define IECM_CTLQ_FLAG_HOST_ID_S 13
+
+#define IECM_CTLQ_FLAG_DD BIT(IECM_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IECM_CTLQ_FLAG_CMP BIT(IECM_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IECM_CTLQ_FLAG_ERR BIT(IECM_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IECM_CTLQ_FLAG_FTYPE_VM BIT(IECM_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IECM_CTLQ_FLAG_FTYPE_PF BIT(IECM_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IECM_CTLQ_FLAG_RD BIT(IECM_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IECM_CTLQ_FLAG_VFC BIT(IECM_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IECM_CTLQ_FLAG_BUF BIT(IECM_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IECM_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IECM_CTLQ_FLAG_HOST_ID_S)
+
+struct iecm_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum iecm_mac_type {
+ IECM_MAC_UNKNOWN = 0,
+ IECM_MAC_PF,
+ IECM_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct iecm_mac_info {
+ enum iecm_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IECM_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum iecm_bus_type {
+ iecm_bus_type_unknown = 0,
+ iecm_bus_type_pci,
+ iecm_bus_type_pcix,
+ iecm_bus_type_pci_express,
+ iecm_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum iecm_bus_speed {
+ iecm_bus_speed_unknown = 0,
+ iecm_bus_speed_33 = 33,
+ iecm_bus_speed_66 = 66,
+ iecm_bus_speed_100 = 100,
+ iecm_bus_speed_120 = 120,
+ iecm_bus_speed_133 = 133,
+ iecm_bus_speed_2500 = 2500,
+ iecm_bus_speed_5000 = 5000,
+ iecm_bus_speed_8000 = 8000,
+ iecm_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum iecm_bus_width {
+ iecm_bus_width_unknown = 0,
+ iecm_bus_width_pcie_x1 = 1,
+ iecm_bus_width_pcie_x2 = 2,
+ iecm_bus_width_pcie_x4 = 4,
+ iecm_bus_width_pcie_x8 = 8,
+ iecm_bus_width_32 = 32,
+ iecm_bus_width_64 = 64,
+ iecm_bus_width_reserved
+};
+
+/* Bus parameters */
+struct iecm_bus_info {
+ enum iecm_bus_speed speed;
+ enum iecm_bus_width width;
+ enum iecm_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct iecm_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct iecm_hw {
+ u8 *hw_addr;
+ u64 hw_addr_len;
+ void *back;
+
+ /* control queue - send and receive */
+ struct iecm_ctlq_info *asq;
+ struct iecm_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct iecm_mac_info mac;
+ struct iecm_bus_info bus;
+ struct iecm_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, iecm_ctlq_info) cq_list_head;
+};
+
+int iecm_ctlq_alloc_ring_res(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq);
+
+void iecm_ctlq_dealloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *iecm_alloc_dma_mem(struct iecm_hw *hw, struct iecm_dma_mem *mem,
+ u64 size);
+void iecm_free_dma_mem(struct iecm_hw *hw, struct iecm_dma_mem *mem);
+#endif /* _IECM_CONTROLQ_H_ */
diff --git a/drivers/net/idpf/base/iecm_controlq_api.h b/drivers/net/idpf/base/iecm_controlq_api.h
new file mode 100644
index 0000000000..27511ffd51
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq_api.h
@@ -0,0 +1,227 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_CONTROLQ_API_H_
+#define _IECM_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "iecm_mem.h"
+#else /* !__KERNEL__ */
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IECM_SUCCESS 0
+#define IECM_ERR_PARAM -53 /* -EBADR */
+#define IECM_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IECM_ERR_NOT_READY -16 /* -EBUSY */
+#define IECM_ERR_BAD_PTR -14 /* -EFAULT */
+#define IECM_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IECM_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IECM_ERR_FW_API_VER -13 /* -EACCESS */
+#define IECM_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IECM_ERR_CFG -22 /* -EINVAL */
+#define IECM_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IECM_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IECM_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IECM_ERR_IN_USE -114 /* -EALREADY */
+#define IECM_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IECM_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IECM_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IECM_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IECM_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IECM_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IECM_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct iecm_hw;
+
+/* Used for queue init, response and events */
+enum iecm_ctlq_type {
+ IECM_CTLQ_TYPE_MAILBOX_TX = 0,
+ IECM_CTLQ_TYPE_MAILBOX_RX = 1,
+ IECM_CTLQ_TYPE_CONFIG_TX = 2,
+ IECM_CTLQ_TYPE_CONFIG_RX = 3,
+ IECM_CTLQ_TYPE_EVENT_RX = 4,
+ IECM_CTLQ_TYPE_RDMA_TX = 5,
+ IECM_CTLQ_TYPE_RDMA_RX = 6,
+ IECM_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct iecm_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct iecm_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IECM_VMVF_TYPE_VF 0
+#define IECM_VMVF_TYPE_VM 1
+#define IECM_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IECM_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IECM_DIRECT_CTX_SIZE 16
+#define IECM_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IECM_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IECM_INDIRECT_CTX_SIZE];
+ struct iecm_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct iecm_ctlq_create_info {
+ enum iecm_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct iecm_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct iecm_ctlq_info {
+ LIST_ENTRY_TYPE(iecm_ctlq_info) cq_list;
+
+ enum iecm_ctlq_type cq_type;
+ int q_id;
+ iecm_lock cq_lock; /* queue lock
+ * iecm_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct iecm_dma_mem desc_ring; /* descriptor ring memory
+ * iecm_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct iecm_dma_mem **rx_buff;
+ struct iecm_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct iecm_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum iecm_mbx_opc {
+ /* iecm_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ iecm_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* iecm_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ iecm_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* iecm_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ iecm_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* iecm_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ iecm_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+int iecm_ctlq_init(struct iecm_hw *hw, u8 num_q,
+ struct iecm_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int iecm_ctlq_add(struct iecm_hw *hw,
+ struct iecm_ctlq_create_info *qinfo,
+ struct iecm_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void iecm_ctlq_remove(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+int iecm_ctlq_send(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq,
+ u16 num_q_msg,
+ struct iecm_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+int iecm_ctlq_recv(struct iecm_ctlq_info *cq, u16 *num_q_msg,
+ struct iecm_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+int iecm_ctlq_clean_sq(struct iecm_ctlq_info *cq, u16 *clean_count,
+ struct iecm_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+int iecm_ctlq_post_rx_buffs(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq,
+ u16 *buff_count,
+ struct iecm_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+int iecm_ctlq_deinit(struct iecm_hw *hw);
+
+#endif /* _IECM_CONTROLQ_API_H_ */
diff --git a/drivers/net/idpf/base/iecm_controlq_setup.c b/drivers/net/idpf/base/iecm_controlq_setup.c
new file mode 100644
index 0000000000..eb6cf7651d
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "iecm_controlq.h"
+
+
+/**
+ * iecm_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+iecm_ctlq_alloc_desc_ring(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct iecm_ctlq_desc);
+
+ cq->desc_ring.va = iecm_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IECM_ERR_NO_MEMORY;
+
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int iecm_ctlq_alloc_bufs(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ return IECM_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct iecm_dma_mem **)
+ iecm_calloc(hw, cq->ring_size,
+ sizeof(struct iecm_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IECM_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct iecm_dma_mem *bi;
+ int num = 1; /* number of iecm_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct iecm_dma_mem *)iecm_calloc(hw, num,
+ sizeof(struct iecm_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = iecm_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IECM_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ iecm_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ }
+ iecm_free(hw, cq->bi.rx_buff);
+
+ return IECM_ERR_NO_MEMORY;
+}
+
+/**
+ * iecm_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void iecm_ctlq_free_desc_ring(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ iecm_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * iecm_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void iecm_ctlq_free_bufs(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ iecm_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ iecm_free(hw, bi);
+}
+
+/**
+ * iecm_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void iecm_ctlq_dealloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_free_bufs(hw, cq);
+ iecm_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * iecm_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int iecm_ctlq_alloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IECM_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = iecm_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = iecm_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto iecm_init_cq_free_ring;
+
+ /* success! */
+ return IECM_SUCCESS;
+
+iecm_init_cq_free_ring:
+ iecm_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/net/idpf/base/iecm_devids.h b/drivers/net/idpf/base/iecm_devids.h
new file mode 100644
index 0000000000..839214cb40
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_devids.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_DEVIDS_H_
+#define _IECM_DEVIDS_H_
+
+/* Vendor ID */
+#define IECM_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IECM_DEV_ID_PF 0x1452
+
+
+
+
+#endif /* _IECM_DEVIDS_H_ */
diff --git a/drivers/net/idpf/base/iecm_lan_pf_regs.h b/drivers/net/idpf/base/iecm_lan_pf_regs.h
new file mode 100644
index 0000000000..c6c460dab0
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_PF_REGS_H_
+#define _IECM_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/net/idpf/base/iecm_lan_txrx.h b/drivers/net/idpf/base/iecm_lan_txrx.h
new file mode 100644
index 0000000000..3e5320975d
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_TXRX_H_
+#define _IECM_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "iecm_osdep.h"
+#endif
+
+enum iecm_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IECM_HASH_INVALID = 0,
+ IECM_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IECM_HASH_NONF_MULTICAST_IPV4_UDP,
+ IECM_HASH_NONF_IPV4_UDP,
+ IECM_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IECM_HASH_NONF_IPV4_TCP,
+ IECM_HASH_NONF_IPV4_SCTP,
+ IECM_HASH_NONF_IPV4_OTHER,
+ IECM_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IECM_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IECM_HASH_NONF_MULTICAST_IPV6_UDP,
+ IECM_HASH_NONF_IPV6_UDP,
+ IECM_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IECM_HASH_NONF_IPV6_TCP,
+ IECM_HASH_NONF_IPV6_SCTP,
+ IECM_HASH_NONF_IPV6_OTHER,
+ IECM_HASH_FRAG_IPV6,
+ IECM_HASH_NONF_RSVD47,
+ IECM_HASH_NONF_FCOE_OX,
+ IECM_HASH_NONF_FCOE_RX,
+ IECM_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IECM_HASH_L2_PAYLOAD = 63,
+ IECM_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IECM_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IECM_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IECM_HASH_FRAG_IPV4) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IECM_HASH_FRAG_IPV6) | \
+ BIT_ULL(IECM_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap belwo comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IECM_DEFAULT_RSS_HASH_EXPANDED (IECM_DEFAULT_RSS_HASH | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IECM_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IECM_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For iecm_splitq_base_tx_compl_desc */
+#define IECM_TXD_COMPLQ_GEN_S 15
+#define IECM_TXD_COMPLQ_GEN_M BIT_ULL(IECM_TXD_COMPLQ_GEN_S)
+#define IECM_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IECM_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IECM_TXD_COMPLQ_COMPL_TYPE_S)
+#define IECM_TXD_COMPLQ_QID_S 0
+#define IECM_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IECM_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IECM_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IECM_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IECM_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IECM_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IECM_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IECM_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IECM_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IECM_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IECM_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IECM_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IECM_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IECM_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_UDP_TUNNELING BIT_ULL(IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_GRE_TUNNELING (0x2ULL << IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IECM_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IECM_TXD_CTX_QW1_MSS_S 50
+#define IECM_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IECM_TXD_CTX_QW1_MSS_S)
+#define IECM_TXD_CTX_QW1_TSO_LEN_S 30
+#define IECM_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IECM_TXD_CTX_QW1_TSO_LEN_S)
+#define IECM_TXD_CTX_QW1_CMD_S 4
+#define IECM_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IECM_TXD_CTX_QW1_CMD_S)
+#define IECM_TXD_CTX_QW1_DTYPE_S 0
+#define IECM_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IECM_TXD_CTX_QW1_DTYPE_S)
+#define IECM_TXD_QW1_L2TAG1_S 48
+#define IECM_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IECM_TXD_QW1_L2TAG1_S)
+#define IECM_TXD_QW1_TX_BUF_SZ_S 34
+#define IECM_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IECM_TXD_QW1_TX_BUF_SZ_S)
+#define IECM_TXD_QW1_OFFSET_S 16
+#define IECM_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IECM_TXD_QW1_OFFSET_S)
+#define IECM_TXD_QW1_CMD_S 4
+#define IECM_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IECM_TXD_QW1_CMD_S)
+#define IECM_TXD_QW1_DTYPE_S 0
+#define IECM_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IECM_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IECM_TXD_COMPLT_ITR_FLUSH 0
+#define IECM_TXD_COMPLT_RULE_MISS 1
+#define IECM_TXD_COMPLT_RS 2
+#define IECM_TXD_COMPLT_REINJECTED 3
+#define IECM_TXD_COMPLT_RE 4
+#define IECM_TXD_COMPLT_SW_MARKER 5
+
+enum iecm_tx_desc_dtype_value {
+ IECM_TX_DESC_DTYPE_DATA = 0,
+ IECM_TX_DESC_DTYPE_CTX = 1,
+ IECM_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IECM_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IECM_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IECM_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IECM_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IECM_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IECM_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IECM_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IECM_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum iecm_tx_ctx_desc_cmd_bits {
+ IECM_TX_CTX_DESC_TSO = 0x01,
+ IECM_TX_CTX_DESC_TSYN = 0x02,
+ IECM_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IECM_TX_CTX_DESC_RSVD = 0x08,
+ IECM_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IECM_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IECM_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IECM_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IECM_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IECM_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IECM_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum iecm_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IECM_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IECM_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IECM_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IECM_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IECM_TX_DESC_LEN_MACLEN_S)
+#define IECM_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IECM_TX_DESC_LEN_IPLEN_S)
+#define IECM_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IECM_TX_DESC_LEN_L4_LEN_S)
+#define IECM_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IECM_TX_DESC_LEN_L4_LEN_S)
+
+enum iecm_tx_base_desc_cmd_bits {
+ IECM_TX_DESC_CMD_EOP = 0x0001,
+ IECM_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IECM_TX_DESC_CMD_ICRC = 0x0004,
+ IECM_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IECM_TX_DESC_CMD_RSVD1 = 0x0010,
+ IECM_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IECM_TX_DESC_CMD_RSVD2 = 0x0080,
+ IECM_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IECM_TX_DESC_CMD_RSVD3 = 0x0400,
+ IECM_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct iecm_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct iecm_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct iecm_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum iecm_tx_flex_desc_cmd_bits {
+ IECM_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IECM_TX_FLEX_DESC_CMD_RS = 0x02,
+ IECM_TX_FLEX_DESC_CMD_RE = 0x04,
+ IECM_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IECM_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IECM_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IECM_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IECM_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct iecm_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IECM_FLEX_TXD_QW1_DTYPE_S 0
+#define IECM_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IECM_FLEX_TXD_QW1_DTYPE_S)
+#define IECM_FLEX_TXD_QW1_CMD_S 5
+#define IECM_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IECM_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IECM_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct iecm_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IECM_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IECM_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IECM_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IECM_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IECM_TXD_FLEX_FLOW_RXR 0x4000
+#define IECM_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum iecm_tx_flex_ctx_desc_cmd_bits {
+ IECM_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IECM_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IECM_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct iecm_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IECM_TXD_FLEX_CTX_TLEN_M 0x1FFFF
+#define IECM_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IECM_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union iecm_flex_tx_ctx_desc {
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct iecm_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct iecm_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IECM_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IECM_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IECM_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IECM_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct iecm_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IECM_TXD_FLEX_CTX_TLEN_S 0
+#define IECM_TXD_FLEX_CTX_TLEN_M 0x1FFFF
+#define IECM_TXD_FLEX_CTX_FNUM_S 18
+#define IECM_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IECM_TXD_FLEX_CTX_HOST_S 29
+#define IECM_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IECM_TXD_FLEX_CTX_MSS_RT_0 0
+#define IECM_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IECM_TXD_FLEX_CTX_FTYPE_S 14
+#define IECM_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IECM_TXD_FLEX_CTX_FTYPE_S)
+#define IECM_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IECM_TXD_FLEX_CTX_FTYPE_S)
+#define IECM_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IECM_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IECM_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IECM_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IECM_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IECM_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_PASID_VALID_S)
+#define IECM_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IECM_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_TPH_S)
+#define IECM_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IECM_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IECM_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IECM_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IECM_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IECM_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IECM_LAN_TXRX_H_ */
diff --git a/drivers/net/idpf/base/iecm_lan_vf_regs.h b/drivers/net/idpf/base/iecm_lan_vf_regs.h
new file mode 100644
index 0000000000..1ba1a8dea6
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_VF_REGS_H_
+#define _IECM_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/net/idpf/base/iecm_prototype.h b/drivers/net/idpf/base/iecm_prototype.h
new file mode 100644
index 0000000000..cd3ee8dcbc
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_PROTOTYPE_H_
+#define _IECM_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "iecm_osdep.h"
+#include "iecm_controlq.h"
+#include "iecm_type.h"
+#include "iecm_alloc.h"
+#include "iecm_devids.h"
+#include "iecm_controlq_api.h"
+#include "iecm_lan_pf_regs.h"
+#include "iecm_lan_vf_regs.h"
+#include "iecm_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int iecm_init_hw(struct iecm_hw *hw, struct iecm_ctlq_size ctlq_size);
+int iecm_deinit_hw(struct iecm_hw *hw);
+
+int iecm_clean_arq_element(struct iecm_hw *hw,
+ struct iecm_arq_event_info *e,
+ u16 *events_pending);
+bool iecm_asq_done(struct iecm_hw *hw);
+bool iecm_check_asq_alive(struct iecm_hw *hw);
+
+int iecm_get_rss_lut(struct iecm_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int iecm_set_rss_lut(struct iecm_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int iecm_get_rss_key(struct iecm_hw *hw, u16 seid,
+ struct iecm_get_set_rss_key_data *key);
+int iecm_set_rss_key(struct iecm_hw *hw, u16 seid,
+ struct iecm_get_set_rss_key_data *key);
+
+int iecm_set_mac_type(struct iecm_hw *hw);
+
+int iecm_reset(struct iecm_hw *hw);
+int iecm_send_msg_to_cp(struct iecm_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IECM_PROTOTYPE_H_ */
diff --git a/drivers/net/idpf/base/iecm_type.h b/drivers/net/idpf/base/iecm_type.h
new file mode 100644
index 0000000000..fdde9c6e61
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_TYPE_H_
+#define _IECM_TYPE_H_
+
+#include "iecm_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p) (_p);
+#define UNREFERENCED_2PARAMETER(_p, _q) (_p); (_q);
+#define UNREFERENCED_3PARAMETER(_p, _q, _r) (_p); (_q); (_r);
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s) (_p); (_q); (_r); (_s);
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t) (_p); (_q); (_r); (_s); (_t);
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct iecm_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct iecm_hw_port_stats {
+ /* eth stats collected by the port */
+ struct iecm_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct iecm_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct iecm_arq_event_info {
+ struct iecm_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct iecm_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct iecm_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct iecm_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IECM_TYPE_H_ */
diff --git a/drivers/net/idpf/base/meson.build b/drivers/net/idpf/base/meson.build
new file mode 100644
index 0000000000..1ad9a87d9d
--- /dev/null
+++ b/drivers/net/idpf/base/meson.build
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = [
+ 'iecm_common.c',
+ 'iecm_controlq.c',
+ 'iecm_controlq_setup.c',
+]
+
+error_cflags = ['-Wno-unused-value',
+ '-Wno-unused-but-set-variable',
+ '-Wno-unused-variable',
+ '-Wno-unused-parameter',
+]
+
+c_args = cflags
+
+foreach flag: error_cflags
+ if cc.has_argument(flag)
+ c_args += flag
+ endif
+endforeach
+
+base_lib = static_library('idpf_base', sources,
+ dependencies: static_rte_eal,
+ c_args: c_args)
+base_objs = base_lib.extract_all_objects()
\ No newline at end of file
diff --git a/drivers/net/idpf/base/siov_regs.h b/drivers/net/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..bb7b2daac0
--- /dev/null
+++ b/drivers/net/idpf/base/siov_regs.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000 /* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000)) /* 2k Rx buffer queues */
+
+#define VDEV_QTX_TAIL_START 0x1100000 /* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000 /* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000)) /* 2k Tx completion queues */
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000) /* Begin at offset of 33MB + 4k to accomdate CTL01 register */
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/net/idpf/base/virtchnl.h b/drivers/net/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..b5d0d5ffd3
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl.h
@@ -0,0 +1,2743 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ **/
+ int count; /* the proto layers must < VIRTCHNL_MAX_NUM_PROTO_HDRS */
+ struct virtchnl_proto_hdr proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/net/idpf/base/virtchnl2.h b/drivers/net/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..43b78f315e
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl2.h
@@ -0,0 +1,1411 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+
+#define VIRTCHNL2_MAX_NUM_PROTO_HDRS 32
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids upto 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is trasmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+struct virtchnl2_proto_hdr {
+ /* see VIRTCHNL2_PROTO_HDR_TYPE definitions */
+ __le32 type;
+ __le32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /*
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL2_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_proto_hdr);
+
+struct virtchnl2_proto_hdrs {
+ u8 tunnel_level;
+ /*
+ * specify where protocol header start from.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ __le32 count; /* the proto layers must < VIRTCHNL2_MAX_NUM_PROTO_HDRS */
+ struct virtchnl2_proto_hdr proto_hdr[VIRTCHNL2_MAX_NUM_PROTO_HDRS];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(2312, virtchnl2_proto_hdrs);
+
+struct virtchnl2_rss_cfg {
+ struct virtchnl2_proto_hdrs proto_hdrs;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ u8 reserved[128];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(2444, virtchnl2_rss_cfg);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to HMA to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * HMA responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to HMA */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting HMA to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to HMA */
+ __le16 mbx_vec_id;
+ /* HMA populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* HMA populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to HMA */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to HMA to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * HMA responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(__le16);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/net/idpf/base/virtchnl2_lan_desc.h b/drivers/net/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..2243b17673
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,603 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ | 16| 0|
+ ----------------------------------------------------------------
+ | RSV | Buffer ID |
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ | 0|
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/net/idpf/base/virtchnl_inline_ipsec.h b/drivers/net/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..902f63bd51
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length caculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 02/13] net/idpf/base: add logs and OS specific implementation
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
2022-08-03 11:30 ` [PATCH 01/13] net/idpf/base: introduce base code Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add PMD logs.
Add some MACRO definations and small functions which are specific
for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/base/iecm_osdep.h | 365 +++++++++++++++++++++++++++++
drivers/net/idpf/idpf_logs.h | 38 +++
2 files changed, 403 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_osdep.h
create mode 100644 drivers/net/idpf/idpf_logs.h
diff --git a/drivers/net/idpf/base/iecm_osdep.h b/drivers/net/idpf/base/iecm_osdep.h
new file mode 100644
index 0000000000..60e21fbc1b
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_osdep.h
@@ -0,0 +1,365 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_OSDEP_H_
+#define _IECM_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#include "../idpf_logs.h"
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum iecm_status iecm_status;
+typedef struct iecm_lock iecm_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0]))
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+#define DEBUGOUT(S) PMD_DRV_LOG_RAW(DEBUG, S)
+#define DEBUGOUT2(S, A...) PMD_DRV_LOG_RAW(DEBUG, S, ##A)
+#define DEBUGFUNC(F) PMD_DRV_LOG_RAW(DEBUG, F)
+
+#define iecm_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "iecm %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define iecm_info(hw, fmt, args...) iecm_debug(hw, IECM_DBG_ALL, fmt, ##args)
+#define iecm_warn(hw, fmt, args...) iecm_debug(hw, IECM_DBG_ALL, fmt, ##args)
+#define iecm_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct iecm_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ iecm_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define iecm_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF iecm_snprintf
+#endif
+
+#define IECM_PCI_REG(reg) rte_read32(reg)
+#define IECM_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IECM_PCI_REG64(reg) rte_read64(reg)
+#define IECM_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define iecm_wmb() rte_io_wmb()
+#define iecm_rmb() rte_io_rmb()
+#define iecm_mb() rte_io_mb()
+
+static inline uint32_t iecm_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IECM_PCI_REG(addr));
+}
+
+static inline uint64_t iecm_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IECM_PCI_REG64(addr));
+}
+
+#define IECM_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IECM_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IECM_READ_REG(hw, reg) iecm_read_addr(IECM_PCI_REG_ADDR((hw), (reg)))
+#define IECM_WRITE_REG(hw, reg, value) \
+ IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) iecm_read_addr(IECM_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) iecm_read_addr64(IECM_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct iecm_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct iecm_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define iecm_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define iecm_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define iecm_free(h, m) rte_free(m)
+
+#define iecm_memset(a, b, c, d) memset((a), (b), (c))
+#define iecm_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define iecm_memdup(a, b, c, d) rte_memcpy(iecm_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct iecm_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+iecm_init_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+iecm_acquire_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+iecm_release_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+iecm_destroy_lock(__attribute__((unused)) struct iecm_lock *sp)
+{
+}
+
+struct iecm_hw;
+
+static inline void *
+iecm_alloc_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
+ struct iecm_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "iecm_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+iecm_free_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
+ struct iecm_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+iecm_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+iecm_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define iecm_usec_delay(x) rte_delay_us(x)
+#define iecm_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IECM_DBG_TRACE
+#define IECM_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IECM_INTEL_VENDOR_ID
+#define IECM_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IECM_OSDEP_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..906aae8463
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,38 @@
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 03/13] net/idpf: support device initialization
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
2022-08-03 11:30 ` [PATCH 01/13] net/idpf/base: introduce base code Junfeng Guo
2022-08-03 11:30 ` [PATCH 02/13] net/idpf/base: add logs and OS specific implementation Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 15:11 ` Stephen Hemminger
` (2 more replies)
2022-08-03 11:30 ` [PATCH 04/13] net/idpf: add queue operations Junfeng Guo
` (10 subsequent siblings)
13 siblings, 3 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li, Xiao Wang
Support device init and the following dev ops:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 651 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 205 +++++++++++
drivers/net/idpf/idpf_vchnl.c | 476 ++++++++++++++++++++++++
drivers/net/idpf/meson.build | 18 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
6 files changed, 1354 insertions(+)
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..87c68226dd
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,651 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define REPRESENTOR "representor"
+
+struct idpf_adapter *adapter;
+uint16_t used_vecs_num;
+
+static const char * const idpf_valid_args[] = {
+ REPRESENTOR,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->next_vport_idx;
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_get_next_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb,
+ uint16_t cur_vport_idx)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ if (cur_vport_idx < max_vport_nb && !vports[cur_vport_idx + 1]) {
+ vport_idx = cur_vport_idx + 1;
+ return vport_idx;
+ }
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ uint16_t idx = adapter->next_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ int i;
+
+ vport->adapter = adapter;
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ adapter->vports[idx] = vport;
+ adapter->cur_vport_nb++;
+ adapter->next_vport_idx = idpf_get_next_vport_idx(adapter->vports,
+ adapter->max_vport_nb, idx);
+ if (adapter->next_vport_idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Failed to get next vport id");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ int ret = 0;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ return ret;
+ }
+
+ ret = idpf_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ return ret;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ return ret;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return ret;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_destroy_vport(vport);
+
+ return 0;
+}
+
+static int idpf_parse_devargs(struct rte_eth_dev *dev)
+{
+ struct rte_devargs *devargs = dev->device->devargs;
+ struct rte_kvargs *kvlist;
+ int ret = 0;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct iecm_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IECM_READ_REG(hw, PFGEN_CTRL);
+ IECM_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct iecm_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IECM_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct iecm_hw *hw)
+{
+ struct iecm_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IECM_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IECM_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct iecm_ctlq_info *ctlq;
+ int ret = 0;
+
+ ret = iecm_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct iecm_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ iecm_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_eth_dev *dev)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ int ret = 0;
+
+ if (adapter->initialized)
+ return 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ ret = idpf_parse_devargs(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to parse devargs");
+ goto err;
+ }
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vport_nb = 0;
+ adapter->next_vport_idx = 0;
+ adapter->initialized = true;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ iecm_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, __rte_unused void *init_params)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ ret = idpf_adapter_init(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return ret;
+ }
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ vport->dev_data = dev->data;
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err;
+ }
+
+err:
+ return ret;
+}
+
+static int
+idpf_dev_uninit(struct rte_eth_dev *dev)
+{
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return -EPERM;
+
+ idpf_dev_close(dev);
+
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IECM_INTEL_VENDOR_ID, IECM_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct rte_eth_devargs eth_da = { .nb_representor_ports = 0 };
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+
+ if (pci_dev->device.devargs) {
+ retval = rte_eth_devargs_parse(pci_dev->device.devargs->args,
+ ð_da);
+ if (retval)
+ return retval;
+ }
+
+ if (!eth_da.nb_representor_ports) {
+ PMD_INIT_LOG(ERR, "Failed to probe, need to add representor devargs.");
+ return -1;
+ }
+
+ if (!adapter) {
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+ }
+
+ for (i = 0; i < eth_da.nb_representor_ports; i++) {
+ snprintf(name, sizeof(name), "idpf_vport_%d",
+ eth_da.representor_ports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ NULL);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to creat vport %d", i);
+ }
+
+ return 0;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ int i;
+
+ iecm_ctlq_deinit(hw);
+
+ if (adapter->caps) {
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+ }
+
+ if (adapter->mbx_resp) {
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+ }
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ if (adapter->vport_req_info[i]) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ if (adapter->vport_recv_info[i]) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ }
+ }
+
+ if (adapter->vports) {
+ /* Needn't free adapter->vports[i] since it's private data */
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+ }
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ if (adapter) {
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ adapter = NULL;
+ }
+
+ return rte_eth_dev_pci_generic_remove(pci_dev, idpf_dev_uninit);
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC |
+ RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..501f772fa8
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,205 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+
+#include "base/iecm_osdep.h"
+#include "base/iecm_type.h"
+#include "base/iecm_devids.h"
+#include "base/iecm_lan_txrx.h"
+#include "base/iecm_lan_pf_regs.h"
+#include "base/virtchnl.h"
+#include "base/virtchnl2.h"
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ struct iecm_hw hw;
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t cur_vport_nb;
+ uint16_t next_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool initialized;
+ bool stopped;
+};
+
+extern struct idpf_adapter *adapter;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_caps(struct idpf_adapter *adapter);
+int idpf_create_vport(__rte_unused struct rte_eth_dev *dev);
+int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..4fc15d5b71
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,476 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "base/iecm_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct iecm_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct iecm_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = iecm_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ iecm_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct iecm_ctlq_msg *ctlq_msg;
+ struct iecm_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct iecm_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct iecm_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct iecm_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct iecm_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ iecm_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = iecm_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = iecm_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ iecm_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_ipf(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ struct iecm_ctlq_msg ctlq_msg;
+ struct iecm_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = iecm_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IECM_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from ipf carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from ipf",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = iecm_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ iecm_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ enum idpf_vc_result result;
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops,
+ args->in_args_size,
+ args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ do {
+ result = idpf_read_msg_from_ipf(adapter,
+ args->out_size,
+ args->out_buffer);
+ if (result == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ }
+ _clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_create_vport(__rte_unused struct rte_eth_dev *dev)
+{
+ uint16_t idx = adapter->next_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_destroy_vport(struct idpf_vport *vport)
+{
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
+
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..3a84162f93
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+subdir('base')
+objs = [base_objs]
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
+
+includes += include_directories('base')
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..b7da224860
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
\ No newline at end of file
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index e35652fe63..8910154544 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 04/13] net/idpf: add queue operations
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 15:16 ` Stephen Hemminger
2022-08-03 11:30 ` [PATCH 05/13] net/idpf: add support to get device information Junfeng Guo
` (9 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 53 +-
drivers/net/idpf/idpf_ethdev.h | 7 +
drivers/net/idpf/idpf_rxtx.c | 1264 ++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 182 +++++
drivers/net/idpf/idpf_vchnl.c | 503 +++++++++++++
drivers/net/idpf/meson.build | 1 +
6 files changed, 2009 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 87c68226dd..b302e42a9c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -12,6 +12,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define REPRESENTOR "representor"
@@ -33,10 +34,18 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static int
-idpf_init_vport_req_info(struct rte_eth_dev *dev)
+idpf_init_vport_req_info(__rte_unused struct rte_eth_dev *dev)
{
struct virtchnl2_create_vport *vport_info;
uint16_t idx = adapter->next_vport_idx;
@@ -193,6 +202,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -203,6 +245,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -211,6 +258,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -228,6 +277,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 501f772fa8..25e0c5cae7 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -200,6 +200,13 @@ int idpf_check_api_version(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
int idpf_create_vport(__rte_unused struct rte_eth_dev *dev);
int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_config_rxqs(struct idpf_vport *vport);
+int idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_config_txqs(struct idpf_vport *vport);
+int idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..7d5428b750
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,1264 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ if (q->complq)
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct iecm_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct iecm_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct iecm_base_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1 =
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct iecm_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ bufq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ bufq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_rx_queue *rxq;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct iecm_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct iecm_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct iecm_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct iecm_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct iecm_base_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct iecm_base_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
+
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+#endif
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc - 1; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->nb_rx_desc - 1);
+ IECM_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->nb_rx_desc - 1);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
+
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..3997082b21
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,182 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "base/iecm_osdep.h"
+#include "base/iecm_type.h"
+#include "base/iecm_devids.h"
+#include "base/iecm_lan_txrx.h"
+#include "base/iecm_lan_pf_regs.h"
+#include "base/virtchnl.h"
+#include "base/virtchnl2.h"
+#include "base/virtchnl2_lan_desc.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 256
+#define IDPF_MAX_TSO_MSS 9668
+#define IDPF_TSO_MAX_SEG UINT8_MAX
+#define IDPF_TX_MAX_MTU_SEG 8
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t crc_len; /* 0 if CRC stripped, 4 otherwise */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct iecm_base_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct iecm_flex_tx_sched_desc *desc_ring;
+ struct iecm_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
+#endif /* _IDPF_RXTX_H_ */
+
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4fc15d5b71..d78903d983 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "base/iecm_prototype.h"
@@ -450,6 +451,508 @@ idpf_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i, j;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 3a84162f93..fce53ef50c 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,6 +12,7 @@ objs = [base_objs]
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 05/13] net/idpf: add support to get device information
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 04/13] net/idpf: add queue operations Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:30 ` [PATCH 06/13] net/idpf: add support to get packet type Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 71 ++++++++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index b302e42a9c..ab991f9181 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -28,6 +28,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
@@ -42,8 +44,77 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
};
+static int
+idpf_dev_info_get(__rte_unused struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_SCATTER |
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT |
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT |
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(__rte_unused struct rte_eth_dev *dev)
{
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 06/13] net/idpf: add support to get packet type
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 05/13] net/idpf: add support to get device information Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:30 ` [PATCH 07/13] net/idpf: add support to update link status Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 ++
drivers/net/idpf/idpf_rxtx.c | 51 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 57 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ab991f9181..cc70e2e03a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -488,6 +489,8 @@ idpf_adapter_init(struct rte_eth_dev *dev)
if (adapter->initialized)
return 0;
+ idpf_set_default_ptype_table(dev);
+
hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
hw->hw_addr_len = pci_dev->mem_resource[0].len;
hw->back = adapter;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 7d5428b750..eae762ced3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,57 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
+static inline uint32_t
+idpf_get_default_pkt_type(uint16_t ptype)
+{
+ static const uint32_t type_table[IDPF_MAX_PKT_TYPE]
+ __rte_cache_aligned = {
+ [1] = RTE_PTYPE_L2_ETHER,
+ [22] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_FRAG,
+ [23] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4,
+ [24] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_UDP,
+ [26] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_TCP,
+ [27] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_SCTP,
+ [28] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV4 | RTE_PTYPE_L4_ICMP,
+ [88] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_FRAG,
+ [89] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6,
+ [90] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP,
+ [92] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_TCP,
+ [93] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_SCTP,
+ [94] = RTE_PTYPE_L2_ETHER | RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_ICMP,
+ };
+
+ return type_table[ptype];
+}
+
+void __rte_cold
+idpf_set_default_ptype_table(struct rte_eth_dev *dev __rte_unused)
+{
+ int i;
+
+ for (i = 0; i < IDPF_MAX_PKT_TYPE; i++)
+ adapter->ptype_tbl[i] = idpf_get_default_pkt_type(i);
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3997082b21..5e57995f8c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -178,5 +178,8 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_default_ptype_table(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 07/13] net/idpf: add support to update link status
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 06/13] net/idpf: add support to get packet type Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:30 ` [PATCH 08/13] net/idpf: add basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 22 ++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 2 ++
2 files changed, 24 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc70e2e03a..668e488843 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -31,6 +31,27 @@ static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
@@ -46,6 +67,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 25e0c5cae7..130bb7ad2d 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -195,6 +195,8 @@ _atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
return !ret;
}
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_check_api_version(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 08/13] net/idpf: add basic Rx/Tx datapath
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 07/13] net/idpf: add support to update link status Junfeng Guo
@ 2022-08-03 11:30 ` Junfeng Guo
2022-08-03 11:31 ` [PATCH 09/13] net/idpf: add support for RSS Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:30 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo, Xiaoyun Li
Add basic RX & TX support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 69 ++-
drivers/net/idpf/idpf_rxtx.c | 896 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 35 ++
3 files changed, 998 insertions(+), 2 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 668e488843..55ec24872e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -14,12 +14,16 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define REPRESENTOR "representor"
struct idpf_adapter *adapter;
uint16_t used_vecs_num;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
REPRESENTOR,
NULL
};
@@ -157,6 +161,30 @@ idpf_init_vport_req_info(__rte_unused struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = dev->data->nb_tx_queues;
+ vport_info->num_tx_complq =
+ dev->data->nb_tx_queues * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = dev->data->nb_tx_queues;
+ vport_info->num_tx_complq = 0;
+ }
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = dev->data->nb_rx_queues;
+ vport_info->num_rx_bufq =
+ dev->data->nb_rx_queues * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = dev->data->nb_rx_queues;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -344,6 +372,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -394,11 +425,31 @@ idpf_dev_close(struct rte_eth_dev *dev)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_DRV_LOG(WARNING, "invalid value:\"%s\" for key:\"%s\", "
+ "value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int idpf_parse_devargs(struct rte_eth_dev *dev)
{
struct rte_devargs *devargs = dev->device->devargs;
struct rte_kvargs *kvlist;
- int ret = 0;
+ int ret;
if (!devargs)
return 0;
@@ -409,6 +460,17 @@ static int idpf_parse_devargs(struct rte_eth_dev *dev)
return -EINVAL;
}
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
+bail:
rte_kvargs_free(kvlist);
return ret;
}
@@ -637,8 +699,11 @@ idpf_dev_init(struct rte_eth_dev *dev, __rte_unused void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
ret = idpf_adapter_init(dev);
if (ret) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index eae762ced3..e6040cece1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1312,4 +1312,900 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_ERR0_QW1 \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_ERR0_QW1) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_16(rx_desc->hash1) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_HASH3_S);
+ }
+
+ return flags;
+}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->nb_rx_hold;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[nb_refill];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+
+ if (nb_refill <= rx_bufq->rx_free_thresh)
+ return;
+
+ if (nb_refill >= nb_desc)
+ nb_refill = nb_desc - 1;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (delta < nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (int i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (int i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IECM_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ pkt_len -= rxq->crc_len;
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ rxm->ol_flags |= pkt_flags;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct iecm_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct iecm_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_GEN_M) >> IECM_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_COMPL_TYPE_M) >> IECM_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_QID_M) >> IECM_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IECM_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IECM_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union iecm_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IECM_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IECM_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IECM_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IECM_TXD_FLEX_CTX_MSS_RT_M);
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct iecm_flex_tx_sched_desc *txr;
+ volatile struct iecm_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ingnore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union iecm_flex_tx_ctx_desc *ctx_desc =
+ (volatile union iecm_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_CS_EN;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IECM_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxe;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxdp->flex_nic_wb.pkt_len)) -
+ rxq->crc_len;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+
+ nb_hold++;
+ rxe = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ rxm = rxe;
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxdp->flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+
+ volatile struct iecm_base_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ if ((txd[desc_to_clean_to].qw1 &
+ rte_cpu_to_le_64(IECM_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1 = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ * support IP -> L4 and IP -> IP -> L4
+ */
+static inline uint64_t
+idpf_set_tso_ctx(struct rte_mbuf *mbuf, union idpf_tx_offload tx_offload)
+{
+ uint64_t ctx_desc = 0;
+ uint32_t cd_cmd, hdr_len, cd_tso_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return ctx_desc;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+
+ cd_cmd = IECM_TX_CTX_DESC_TSO;
+ cd_tso_len = mbuf->pkt_len - hdr_len;
+ ctx_desc |= ((uint64_t)cd_cmd << IECM_TXD_CTX_QW1_CMD_S) |
+ ((uint64_t)cd_tso_len << IECM_TXD_CTX_QW1_TSO_LEN_S) |
+ ((uint64_t)mbuf->tso_segsz << IECM_TXD_CTX_QW1_MSS_S);
+
+ return ctx_desc;
+}
+
+/* Construct the tx flags */
+static inline uint64_t
+idpf_build_ctob(uint32_t td_cmd, uint32_t td_offset, unsigned int size)
+{
+ return rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)td_cmd << IECM_TXD_QW1_CMD_S) |
+ ((uint64_t)td_offset <<
+ IECM_TXD_QW1_OFFSET_S) |
+ ((uint64_t)size <<
+ IECM_TXD_QW1_TX_BUF_SZ_S));
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct iecm_base_tx_desc *txd;
+ volatile struct iecm_base_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint32_t td_offset;
+ uint64_t ol_flags;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t nb_ctx;
+ uint32_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+ td_offset = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ /* According to datasheet, the bit2 is reserved and must be
+ * set to 1.
+ */
+ td_cmd |= 0x04;
+
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union iecm_flex_tx_ctx_desc *ctx_txd =
+ (volatile union iecm_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1 = idpf_build_ctob(td_cmd, td_offset, slen);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IECM_TX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IECM_TX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1 |=
+ rte_cpu_to_le_64(((uint64_t)td_cmd) <<
+ IECM_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IECM_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
+ if (!m->pkt_len) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ return;
+ } else {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ return;
+ }
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 5e57995f8c..3e79c0ac2a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -42,6 +42,25 @@
#define IDPF_TSO_MAX_SEG UINT8_MAX
#define IDPF_TX_MAX_MTU_SEG 8
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -176,8 +195,24 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
void idpf_set_default_ptype_table(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 09/13] net/idpf: add support for RSS
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-08-03 11:30 ` [PATCH 08/13] net/idpf: add basic Rx/Tx datapath Junfeng Guo
@ 2022-08-03 11:31 ` Junfeng Guo
2022-08-03 11:31 ` [PATCH 10/13] net/idpf: add mtu configuration Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:31 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 108 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 18 ++++++
drivers/net/idpf/idpf_vchnl.c | 93 ++++++++++++++++++++++++++++
3 files changed, 219 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 55ec24872e..0b03ec860a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -85,6 +85,7 @@ idpf_dev_info_get(__rte_unused struct rte_eth_dev *dev, struct rte_eth_dev_info
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -293,6 +294,91 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->num_rx_q;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IECM_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
@@ -300,6 +386,10 @@ idpf_dev_configure(struct rte_eth_dev *dev)
(struct idpf_vport *)dev->data->dev_private;
int ret = 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
ret = idpf_init_vport_req_info(dev);
if (ret) {
PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
@@ -321,6 +411,14 @@ idpf_dev_configure(struct rte_eth_dev *dev)
rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
&dev->data->mac_addrs[0]);
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
return ret;
}
@@ -422,6 +520,16 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_destroy_vport(vport);
+ if (vport->rss_lut) {
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+ }
+
+ if (vport->rss_key) {
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+ }
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 130bb7ad2d..22096c4185 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -47,6 +47,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -202,6 +216,9 @@ int idpf_check_api_version(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
int idpf_create_vport(__rte_unused struct rte_eth_dev *dev);
int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_set_rss_key(struct idpf_vport *vport);
+int idpf_set_rss_lut(struct idpf_vport *vport);
+int idpf_set_rss_hash(struct idpf_vport *vport);
int idpf_config_rxqs(struct idpf_vport *vport);
int idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_config_txqs(struct idpf_vport *vport);
@@ -212,3 +229,4 @@ int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
#endif /* _IDPF_ETHDEV_H_ */
+
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index d78903d983..07b220016f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -451,6 +451,99 @@ idpf_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_set_rss_key(struct idpf_vport *vport)
+{
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_set_rss_lut(struct idpf_vport *vport)
+{
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_set_rss_hash(struct idpf_vport *vport)
+{
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_config_rxqs(struct idpf_vport *vport)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 10/13] net/idpf: add mtu configuration
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-08-03 11:31 ` [PATCH 09/13] net/idpf: add support for RSS Junfeng Guo
@ 2022-08-03 11:31 ` Junfeng Guo
2022-08-03 11:31 ` [PATCH 11/13] net/idpf: add hw statistics Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:31 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0b03ec860a..4b8e3c4d5b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -34,6 +34,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
int
idpf_dev_link_update(struct rte_eth_dev *dev,
@@ -72,6 +73,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -143,6 +145,18 @@ idpf_dev_info_get(__rte_unused struct rte_eth_dev *dev, struct rte_eth_dev_info
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(__rte_unused struct rte_eth_dev *dev)
{
@@ -465,6 +479,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 11/13] net/idpf: add hw statistics
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-08-03 11:31 ` [PATCH 10/13] net/idpf: add mtu configuration Junfeng Guo
@ 2022-08-03 11:31 ` Junfeng Guo
2022-08-03 11:31 ` [PATCH 12/13] net/idpf: support write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:31 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo, Mingxia Liu
Add hardware packets/bytes statisticsi query and reset.
Signed-off-by: Mingxia Liu <mingxia.liu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 82 +++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_vchnl.c | 26 +++++++++++
3 files changed, 109 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4b8e3c4d5b..b934488d0b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,9 @@ static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
+static int idpf_dev_stats_get(struct rte_eth_dev *dev,
+ struct rte_eth_stats *stats);
+static int idpf_dev_stats_reset(struct rte_eth_dev *dev);
int
idpf_dev_link_update(struct rte_eth_dev *dev,
@@ -74,6 +77,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
.mtu_set = idpf_dev_mtu_set,
+ .stats_get = idpf_dev_stats_get,
+ .stats_reset = idpf_dev_stats_reset,
};
static int
@@ -157,8 +162,79 @@ idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
return 0;
}
+static void
+idpf_stat_update(uint64_t *offset, uint64_t *stat)
+{
+ *stat = *stat - *offset;
+}
+
+
+static void
+idpf_update_stats(struct virtchnl2_vport_stats *oes, struct virtchnl2_vport_stats *nes)
+{
+ idpf_stat_update(&oes->rx_bytes, &nes->rx_bytes);
+ idpf_stat_update(&oes->rx_unicast, &nes->rx_unicast);
+ idpf_stat_update(&oes->rx_multicast, &nes->rx_multicast);
+ idpf_stat_update(&oes->rx_broadcast, &nes->rx_broadcast);
+ idpf_stat_update(&oes->rx_discards, &nes->rx_discards);
+ idpf_stat_update(&oes->tx_bytes, &nes->tx_bytes);
+ idpf_stat_update(&oes->tx_unicast, &nes->tx_unicast);
+ idpf_stat_update(&oes->tx_multicast, &nes->tx_multicast);
+ idpf_stat_update(&oes->tx_broadcast, &nes->tx_broadcast);
+ idpf_stat_update(&oes->tx_errors, &nes->tx_errors);
+ idpf_stat_update(&oes->tx_discards, &nes->tx_discards);
+}
+
static int
-idpf_init_vport_req_info(__rte_unused struct rte_eth_dev *dev)
+idpf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct virtchnl2_vport_stats *pstats = NULL;
+ int ret;
+
+ ret = idpf_query_stats(vport, &pstats);
+ if (ret == 0) {
+ uint8_t crc_stats_len = (dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_KEEP_CRC) ? 0 :
+ RTE_ETHER_CRC_LEN;
+ idpf_update_stats(&vport->eth_stats_offset, pstats);
+ stats->ipackets = pstats->rx_unicast + pstats->rx_multicast +
+ pstats->rx_broadcast - pstats->rx_discards;
+ stats->opackets = pstats->tx_broadcast + pstats->tx_multicast +
+ pstats->tx_unicast;
+ stats->imissed = pstats->rx_discards;
+ stats->oerrors = pstats->tx_errors + pstats->tx_discards;
+ stats->ibytes = pstats->rx_bytes;
+ stats->ibytes -= stats->ipackets * crc_stats_len;
+ stats->obytes = pstats->tx_bytes;
+ } else {
+ PMD_DRV_LOG(ERR, "Get statistics failed");
+ }
+ return ret;
+}
+
+
+static int
+idpf_dev_stats_reset(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct virtchnl2_vport_stats *pstats = NULL;
+ int ret;
+
+ ret = idpf_query_stats(vport, &pstats);
+ if (ret != 0)
+ return ret;
+
+ /* set stats offset base on current values */
+ vport->eth_stats_offset = *pstats;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
struct virtchnl2_create_vport *vport_info;
uint16_t idx = adapter->next_vport_idx;
@@ -499,6 +575,10 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_vport;
}
+ if (idpf_dev_stats_reset(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to reset stats");
+ goto err_vport;
+ }
return 0;
err_vport:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 22096c4185..c0cbf4c3c6 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -227,6 +227,8 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_query_stats(struct idpf_vport *vport,
+ struct virtchnl2_vport_stats **pstats);
#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 07b220016f..563f8f649e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -219,6 +219,7 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_GET_STATS:
/* for init virtchnl ops, need to poll the response */
do {
result = idpf_read_msg_from_ipf(adapter,
@@ -1070,3 +1071,28 @@ idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+int
+idpf_query_stats(struct idpf_vport *vport,
+ struct virtchnl2_vport_stats **pstats)
+{
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = VIRTCHNL2_OP_GET_STATS;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_STATS");
+ *pstats = NULL;
+ return err;
+ }
+ *pstats = (struct virtchnl2_vport_stats *)args.out_buffer;
+ return 0;
+}
+
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 12/13] net/idpf: support write back based on ITR expire
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-08-03 11:31 ` [PATCH 11/13] net/idpf: add hw statistics Junfeng Guo
@ 2022-08-03 11:31 ` Junfeng Guo
2022-08-03 11:31 ` [PATCH 13/13] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:31 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 117 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 108 ++++++++++++++++++++++++++++++
3 files changed, 233 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index b934488d0b..1e9564a5a9 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -512,6 +512,87 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport =
+ (struct idpf_vport *)dev->data->dev_private;
+ struct virtchnl2_queue_vector *qv_map;
+ struct iecm_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start = vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start = vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IECM_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x", dynctl_val);
+ itrn_val = IECM_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are writen back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IECM_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IECM_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -550,6 +631,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport =
(struct idpf_vport *)dev->data->dev_private;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -562,6 +646,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -603,6 +704,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -631,6 +738,16 @@ idpf_dev_close(struct rte_eth_dev *dev)
vport->rss_key = NULL;
}
+ if (vport->recv_vectors) {
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+ }
+
+ if (vport->qv_map) {
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+ }
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c0cbf4c3c6..32520c03bb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -119,6 +119,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -229,6 +234,9 @@ int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_query_stats(struct idpf_vport *vport,
struct virtchnl2_vport_stats **pstats);
+int idpf_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_dealloc_vectors(struct idpf_vport *vport);
#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 563f8f649e..bfb3b08465 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -220,6 +220,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
case VIRTCHNL2_OP_GET_STATS:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
do {
result = idpf_read_msg_from_ipf(adapter,
@@ -900,6 +904,110 @@ idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH 13/13] net/idpf: add AVX512 data path for single queue model
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-08-03 11:31 ` [PATCH 12/13] net/idpf: support write back based on ITR expire Junfeng Guo
@ 2022-08-03 11:31 ` Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-08-03 11:31 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 5 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 137 ++++
drivers/net/idpf/idpf_rxtx.h | 8 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 917 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 85 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1185 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1e9564a5a9..590f1b25aa 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -147,6 +147,11 @@ idpf_dev_info_get(__rte_unused struct rte_eth_dev *dev, struct rte_eth_dev_info
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 32520c03bb..7ad60f36a8 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -164,6 +164,11 @@ struct idpf_adapter {
bool initialized;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
extern struct idpf_adapter *adapter;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e6040cece1..af23857f2a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2177,16 +2179,111 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport =
(struct idpf_vport *)dev->data->dev_private;
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+#ifdef RTE_ARCH_X86
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif
+ }
+#endif
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
return;
}
@@ -2197,12 +2294,52 @@ idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport =
(struct idpf_vport *)dev->data->dev_private;
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_tx_queue *txq;
+ int i;
+
+#ifdef RTE_ARCH_X86
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif
+ }
+#endif
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3e79c0ac2a..dad048a6dc 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -114,6 +114,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -205,8 +209,12 @@ uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
void idpf_stop_queues(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..70eb5e1e7e
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,917 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 2 features are supported in RX path,
+ * 1, checksum offload
+ * 2, VLAN/QINQ stripping
+ * 3, RSS hash
+ * 4, packet type analysis
+ * 5, flow director ID report
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+#else
+ const __m512i permute_idx = _mm512_set_epi64(7, 3, 6, 2, 5, 1, 4, 0);
+#endif
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+#else
+ /* permute leaves desc 4-7 addresses in header address slots 0-3
+ * but these are ignored by driver since header split not
+ * enabled.
+ */
+ const __m512i desc0_3 = _mm512_permutexvar_epi64(permute_idx,
+ iova_addrs);
+ const __m512i desc4_7 = _mm512_bsrli_epi128(desc0_3, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_7);
+#endif
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* constants used in processing loop */
+ const __m512i crc_adjust =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 2nd descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 3rd descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 4th descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0 /* ignore pkt_type field */
+ );
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the above crc and shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+ mb4_7 = _mm512_add_epi32(mb4_7, crc_adjust);
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+ mb0_3 = _mm512_add_epi32(mb0_3, crc_adjust);
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value in vlan0.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1 &
+ rte_cpu_to_le_64(IECM_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline void
+idpf_vtx1(volatile struct iecm_base_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)flags << IECM_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IECM_TXD_QW1_TX_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct iecm_base_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)flags << IECM_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct iecm_base_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ /* bit2 is reserved and must be set to 1 according to Spec */
+ uint64_t flags = IECM_TX_DESC_CMD_EOP | 0x04;
+ uint64_t rs = IECM_TX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1 |=
+ rte_cpu_to_le_64(((uint64_t)IECM_TX_DESC_CMD_RS) <<
+ IECM_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IECM_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..c60ffe9cbc
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index fce53ef50c..403f5e45a0 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -16,4 +16,32 @@ sources = files(
'idpf_vchnl.c',
)
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
+
includes += include_directories('base')
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH 03/13] net/idpf: support device initialization
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
@ 2022-08-03 15:11 ` Stephen Hemminger
2022-08-08 4:43 ` Guo, Junfeng
2022-10-31 18:00 ` Ali Alnubani
2022-11-02 15:31 ` Raslan Darawsheh
2 siblings, 1 reply; 376+ messages in thread
From: Stephen Hemminger @ 2022-08-03 15:11 UTC (permalink / raw)
To: Junfeng Guo
Cc: qi.z.zhang, jingjing.wu, beilei.xing, dev, Xiaoyun Li, Xiao Wang
On Wed, 3 Aug 2022 19:30:54 +0800
Junfeng Guo <junfeng.guo@intel.com> wrote:
> +
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
Cast of void pointer dev_private is unnecessary in C (only in C++ is it needed).
> + int ret = 0;
Useless initialization, you are setting in next line.
> +
> + ret = idpf_init_vport_req_info(dev);
> + if (ret) {
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH 04/13] net/idpf: add queue operations
2022-08-03 11:30 ` [PATCH 04/13] net/idpf: add queue operations Junfeng Guo
@ 2022-08-03 15:16 ` Stephen Hemminger
2022-08-08 4:44 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Stephen Hemminger @ 2022-08-03 15:16 UTC (permalink / raw)
To: Junfeng Guo; +Cc: qi.z.zhang, jingjing.wu, beilei.xing, dev, Xiaoyun Li
On Wed, 3 Aug 2022 19:30:55 +0800
Junfeng Guo <junfeng.guo@intel.com> wrote:
> +static void
> +idpf_tx_queue_release(void *txq)
> +{
> + struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
> +
> + if (!q)
> + return;
> +
> + if (q->complq)
> + rte_free(q->complq);
null pointer check before calling rte_free is unnecessary.
rte_free(NULL) works.
There is even a script used to scan DPDK to fix this.
devtools/cocci/nullfree.cocci
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 03/13] net/idpf: support device initialization
2022-08-03 15:11 ` Stephen Hemminger
@ 2022-08-08 4:43 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-08-08 4:43 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Zhang, Qi Z, Wu, Jingjing, Xing, Beilei, dev, Li, Xiaoyun, Wang, Xiao W
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Wednesday, August 3, 2022 23:12
> To: Guo, Junfeng <junfeng.guo@intel.com>
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>;
> dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>
> Subject: Re: [PATCH 03/13] net/idpf: support device initialization
>
> On Wed, 3 Aug 2022 19:30:54 +0800
> Junfeng Guo <junfeng.guo@intel.com> wrote:
>
> > +
> > +static int
> > +idpf_dev_configure(struct rte_eth_dev *dev)
> > +{
> > + struct idpf_vport *vport =
> > + (struct idpf_vport *)dev->data->dev_private;
>
> Cast of void pointer dev_private is unnecessary in C (only in C++ is it
> needed).
>
> > + int ret = 0;
>
> Useless initialization, you are setting in next line.
Thanks for your review! Will refine this in the patchset of upcoming version.
>
> > +
> > + ret = idpf_init_vport_req_info(dev);
> > + if (ret) {
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 04/13] net/idpf: add queue operations
2022-08-03 15:16 ` Stephen Hemminger
@ 2022-08-08 4:44 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-08-08 4:44 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Zhang, Qi Z, Wu, Jingjing, Xing, Beilei, dev, Li, Xiaoyun
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Wednesday, August 3, 2022 23:16
> To: Guo, Junfeng <junfeng.guo@intel.com>
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>;
> dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: Re: [PATCH 04/13] net/idpf: add queue operations
>
> On Wed, 3 Aug 2022 19:30:55 +0800
> Junfeng Guo <junfeng.guo@intel.com> wrote:
>
> > +static void
> > +idpf_tx_queue_release(void *txq)
> > +{
> > + struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
> > +
> > + if (!q)
> > + return;
> > +
> > + if (q->complq)
> > + rte_free(q->complq);
>
> null pointer check before calling rte_free is unnecessary.
> rte_free(NULL) works.
Thanks for your review! Will refine this in the patchset of upcoming version.
>
> There is even a script used to scan DPDK to fix this.
> devtools/cocci/nullfree.cocci
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 00/14] add support for idpf PMD in DPDK
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-08-03 11:31 ` [PATCH 13/13] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 01/14] net/idpf/base: introduce base code Junfeng Guo
` (15 more replies)
13 siblings, 16 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD [*EXPERIMENTAL*] in DPDK for Intel Device ID of 0x1452.
v2:
fixed some coding style issues and did some refactors.
Junfeng Guo (14):
net/idpf/base: introduce base code
net/idpf/base: add logs and OS specific implementation
net/idpf: add support for device initialization
net/idpf: add support for queue operations
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for link status update
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for RSS
net/idpf: add support for mtu configuration
net/idpf: add support for hw statistics
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
drivers/net/idpf/base/iecm_alloc.h | 22 +
drivers/net/idpf/base/iecm_common.c | 359 +++
drivers/net/idpf/base/iecm_controlq.c | 662 ++++
drivers/net/idpf/base/iecm_controlq.h | 214 ++
drivers/net/idpf/base/iecm_controlq_api.h | 227 ++
drivers/net/idpf/base/iecm_controlq_setup.c | 179 ++
drivers/net/idpf/base/iecm_devids.h | 18 +
drivers/net/idpf/base/iecm_lan_pf_regs.h | 134 +
drivers/net/idpf/base/iecm_lan_txrx.h | 428 +++
drivers/net/idpf/base/iecm_lan_vf_regs.h | 114 +
drivers/net/idpf/base/iecm_osdep.h | 365 +++
drivers/net/idpf/base/iecm_prototype.h | 45 +
drivers/net/idpf/base/iecm_type.h | 106 +
drivers/net/idpf/base/meson.build | 27 +
drivers/net/idpf/base/siov_regs.h | 41 +
drivers/net/idpf/base/virtchnl.h | 2743 +++++++++++++++++
drivers/net/idpf/base/virtchnl2.h | 1411 +++++++++
drivers/net/idpf/base/virtchnl2_lan_desc.h | 603 ++++
drivers/net/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/net/idpf/idpf_ethdev.c | 1341 ++++++++
drivers/net/idpf/idpf_ethdev.h | 270 ++
drivers/net/idpf/idpf_logs.h | 38 +
drivers/net/idpf/idpf_rxtx.c | 2393 ++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 322 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 917 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 89 +
drivers/net/idpf/idpf_vchnl.c | 1470 +++++++++
drivers/net/idpf/meson.build | 48 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
30 files changed, 15157 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_alloc.h
create mode 100644 drivers/net/idpf/base/iecm_common.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_api.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_setup.c
create mode 100644 drivers/net/idpf/base/iecm_devids.h
create mode 100644 drivers/net/idpf/base/iecm_lan_pf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_lan_txrx.h
create mode 100644 drivers/net/idpf/base/iecm_lan_vf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_osdep.h
create mode 100644 drivers/net/idpf/base/iecm_prototype.h
create mode 100644 drivers/net/idpf/base/iecm_type.h
create mode 100644 drivers/net/idpf/base/meson.build
create mode 100644 drivers/net/idpf/base/siov_regs.h
create mode 100644 drivers/net/idpf/base/virtchnl.h
create mode 100644 drivers/net/idpf/base/virtchnl2.h
create mode 100644 drivers/net/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/net/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 01/14] net/idpf/base: introduce base code
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 13:20 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation Junfeng Guo
` (14 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Introduce base code for IDPF (Infrastructure Data Path Function) PMD.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/base/iecm_alloc.h | 22 +
drivers/net/idpf/base/iecm_common.c | 359 +++
drivers/net/idpf/base/iecm_controlq.c | 662 ++++
drivers/net/idpf/base/iecm_controlq.h | 214 ++
drivers/net/idpf/base/iecm_controlq_api.h | 227 ++
drivers/net/idpf/base/iecm_controlq_setup.c | 179 ++
drivers/net/idpf/base/iecm_devids.h | 18 +
drivers/net/idpf/base/iecm_lan_pf_regs.h | 134 +
drivers/net/idpf/base/iecm_lan_txrx.h | 428 +++
drivers/net/idpf/base/iecm_lan_vf_regs.h | 114 +
drivers/net/idpf/base/iecm_prototype.h | 45 +
drivers/net/idpf/base/iecm_type.h | 106 +
drivers/net/idpf/base/meson.build | 27 +
drivers/net/idpf/base/siov_regs.h | 41 +
drivers/net/idpf/base/virtchnl.h | 2743 +++++++++++++++++
drivers/net/idpf/base/virtchnl2.h | 1411 +++++++++
drivers/net/idpf/base/virtchnl2_lan_desc.h | 603 ++++
drivers/net/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
18 files changed, 7900 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_alloc.h
create mode 100644 drivers/net/idpf/base/iecm_common.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.c
create mode 100644 drivers/net/idpf/base/iecm_controlq.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_api.h
create mode 100644 drivers/net/idpf/base/iecm_controlq_setup.c
create mode 100644 drivers/net/idpf/base/iecm_devids.h
create mode 100644 drivers/net/idpf/base/iecm_lan_pf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_lan_txrx.h
create mode 100644 drivers/net/idpf/base/iecm_lan_vf_regs.h
create mode 100644 drivers/net/idpf/base/iecm_prototype.h
create mode 100644 drivers/net/idpf/base/iecm_type.h
create mode 100644 drivers/net/idpf/base/meson.build
create mode 100644 drivers/net/idpf/base/siov_regs.h
create mode 100644 drivers/net/idpf/base/virtchnl.h
create mode 100644 drivers/net/idpf/base/virtchnl2.h
create mode 100644 drivers/net/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/net/idpf/base/virtchnl_inline_ipsec.h
diff --git a/drivers/net/idpf/base/iecm_alloc.h b/drivers/net/idpf/base/iecm_alloc.h
new file mode 100644
index 0000000000..7ea219c784
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_ALLOC_H_
+#define _IECM_ALLOC_H_
+
+/* Memory types */
+enum iecm_memset_type {
+ IECM_NONDMA_MEM = 0,
+ IECM_DMA_MEM
+};
+
+/* Memcpy types */
+enum iecm_memcpy_type {
+ IECM_NONDMA_TO_NONDMA = 0,
+ IECM_NONDMA_TO_DMA,
+ IECM_DMA_TO_DMA,
+ IECM_DMA_TO_NONDMA
+};
+
+#endif /* _IECM_ALLOC_H_ */
diff --git a/drivers/net/idpf/base/iecm_common.c b/drivers/net/idpf/base/iecm_common.c
new file mode 100644
index 0000000000..418fd99298
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_common.c
@@ -0,0 +1,359 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "iecm_type.h"
+#include "iecm_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * iecm_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int iecm_set_mac_type(struct iecm_hw *hw)
+{
+ int status = IECM_SUCCESS;
+
+ DEBUGFUNC("iecm_set_mac_type\n");
+
+ if (hw->vendor_id == IECM_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IECM_DEV_ID_PF:
+ hw->mac.type = IECM_MAC_PF;
+ break;
+ default:
+ hw->mac.type = IECM_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IECM_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("iecm_set_mac_type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * iecm_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int iecm_init_hw(struct iecm_hw *hw, struct iecm_ctlq_size ctlq_size)
+{
+ struct iecm_ctlq_create_info *q_info;
+ int status = IECM_SUCCESS;
+ struct iecm_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct iecm_ctlq_create_info *)
+ iecm_calloc(hw, 2, sizeof(struct iecm_ctlq_create_info));
+ if (!q_info)
+ return IECM_ERR_NO_MEMORY;
+
+ q_info[0].type = IECM_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IECM_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IECM_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IECM_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = iecm_ctlq_init(hw, 2, q_info);
+ if (status != IECM_SUCCESS) {
+ /* TODO return error */
+ iecm_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, iecm_ctlq_info, cq_list) {
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. iecm_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int iecm_send_msg_to_cp(struct iecm_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct iecm_ctlq_msg ctlq_msg = { 0 };
+ struct iecm_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = iecm_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct iecm_dma_mem *)
+ iecm_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IECM_ERR_NO_MEMORY;
+
+ iecm_memcpy(dma_mem.va, msg, msglen, IECM_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = iecm_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ iecm_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * iecm_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool iecm_asq_done(struct iecm_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * iecm_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool iecm_check_asq_alive(struct iecm_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * iecm_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int iecm_clean_arq_element(struct iecm_hw *hw,
+ struct iecm_arq_event_info *e, u16 *pending)
+{
+ struct iecm_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = iecm_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ iecm_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IECM_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * iecm_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int iecm_deinit_hw(struct iecm_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return iecm_ctlq_deinit(hw);
+}
+
+/**
+ * iecm_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int iecm_reset(struct iecm_hw *hw)
+{
+ return iecm_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IECM_SUCCESS, NULL, 0);
+}
+
+/**
+ * iecm_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int iecm_get_set_rss_lut(struct iecm_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int iecm_get_rss_lut(struct iecm_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return iecm_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * iecm_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int iecm_set_rss_lut(struct iecm_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return iecm_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * iecm_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int iecm_get_set_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int iecm_get_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key)
+{
+ return iecm_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * iecm_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int iecm_set_rss_key(struct iecm_hw *hw, u16 vsi_id,
+ struct iecm_get_set_rss_key_data *key)
+{
+ return iecm_get_set_rss_key(hw, vsi_id, key, true);
+}
diff --git a/drivers/net/idpf/base/iecm_controlq.c b/drivers/net/idpf/base/iecm_controlq.c
new file mode 100644
index 0000000000..49caa624ca
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq.c
@@ -0,0 +1,662 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "iecm_controlq.h"
+
+/**
+ * iecm_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+iecm_ctlq_setup_regs(struct iecm_ctlq_info *cq,
+ struct iecm_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * iecm_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void iecm_ctlq_init_regs(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IECM_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IECM_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * iecm_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void iecm_ctlq_init_rxq_bufs(struct iecm_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct iecm_ctlq_desc *desc = IECM_CTLQ_DESC(cq, i);
+ struct iecm_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IECM_CTLQ_FLAG_BUF | IECM_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * iecm_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void iecm_ctlq_shutdown(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ iecm_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ iecm_release_lock(&cq->cq_lock);
+ iecm_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * iecm_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: iecm_ctlq_init must be called prior to any calls to iecm_ctlq_add
+ */
+int iecm_ctlq_add(struct iecm_hw *hw,
+ struct iecm_ctlq_create_info *qinfo,
+ struct iecm_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IECM_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IECM_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IECM_CTLQ_MAX_BUF_LEN)
+ return IECM_ERR_CFG;
+
+ *cq_out = (struct iecm_ctlq_info *)
+ iecm_calloc(hw, 1, sizeof(struct iecm_ctlq_info));
+ if (!(*cq_out))
+ return IECM_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IECM_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+ /* fallthrough */
+ case IECM_CTLQ_TYPE_MAILBOX_TX:
+ status = iecm_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IECM_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ iecm_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct iecm_ctlq_msg **)
+ iecm_calloc(hw, qinfo->len,
+ sizeof(struct iecm_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IECM_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ iecm_ctlq_setup_regs(*cq_out, qinfo);
+
+ iecm_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ iecm_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ iecm_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void iecm_ctlq_remove(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ iecm_ctlq_shutdown(hw, cq);
+ iecm_free(hw, cq);
+}
+
+/**
+ * iecm_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int iecm_ctlq_init(struct iecm_hw *hw, u8 num_q,
+ struct iecm_ctlq_create_info *q_info)
+{
+ struct iecm_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IECM_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct iecm_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = iecm_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ iecm_ctlq_info, cq_list)
+ iecm_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * iecm_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int iecm_ctlq_deinit(struct iecm_hw *hw)
+{
+ struct iecm_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IECM_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ iecm_ctlq_info, cq_list)
+ iecm_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * iecm_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int iecm_ctlq_send(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ u16 num_q_msg, struct iecm_ctlq_msg q_msg[])
+{
+ struct iecm_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IECM_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IECM_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IECM_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct iecm_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IECM_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IECM_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IECM_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IECM_HOST_ID_MASK) <<
+ IECM_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct iecm_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IECM_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IECM_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(buff->pa));
+
+ iecm_memcpy(&desc->params, msg->ctx.indirect.context,
+ IECM_INDIRECT_CTX_SIZE, IECM_NONDMA_TO_DMA);
+ } else {
+ iecm_memcpy(&desc->params, msg->ctx.direct,
+ IECM_DIRECT_CTX_SIZE, IECM_NONDMA_TO_DMA);
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ iecm_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ iecm_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int iecm_ctlq_clean_sq(struct iecm_ctlq_info *cq, u16 *clean_count,
+ struct iecm_ctlq_msg *msg_status[])
+{
+ struct iecm_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IECM_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IECM_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IECM_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IECM_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ iecm_memset(desc, 0, sizeof(*desc), IECM_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ iecm_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * iecm_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int iecm_ctlq_post_rx_buffs(struct iecm_hw *hw, struct iecm_ctlq_info *cq,
+ u16 *buff_count, struct iecm_dma_mem **buffs)
+{
+ struct iecm_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IECM_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ iecm_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IECM_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IECM_CTLQ_FLAG_BUF | IECM_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IECM_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IECM_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ iecm_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * iecm_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int iecm_ctlq_recv(struct iecm_ctlq_info *cq, u16 *num_q_msg,
+ struct iecm_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct iecm_ctlq_desc *desc;
+ int ret_code = IECM_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IECM_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IECM_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IECM_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ iecm_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IECM_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IECM_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IECM_CTLQ_FLAG_FTYPE_VM |
+ IECM_CTLQ_FLAG_FTYPE_PF)) >>
+ IECM_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IECM_CTLQ_FLAG_ERR)
+ ret_code = IECM_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ iecm_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IECM_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ iecm_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IECM_INDIRECT_CTX_SIZE,
+ IECM_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ iecm_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IECM_DIRECT_CTX_SIZE,
+ IECM_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ iecm_memset(desc, 0, sizeof(struct iecm_ctlq_desc),
+ IECM_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ iecm_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IECM_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/net/idpf/base/iecm_controlq.h b/drivers/net/idpf/base/iecm_controlq.h
new file mode 100644
index 0000000000..0964146b49
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq.h
@@ -0,0 +1,214 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_CONTROLQ_H_
+#define _IECM_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "iecm_osdep.h"
+#include "iecm_alloc.h"
+/* This is used to explicitly annotate when a switch case falls through to the
+ * next case.
+ */
+#define fallthrough do {} while (0)
+#endif
+#include "iecm_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IECM_CTLQ_MAX_RING_SIZE 1024
+#define IECM_CTLQ_MAX_BUF_LEN 4096
+
+#define IECM_CTLQ_DESC(R, i) \
+ (&(((struct iecm_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IECM_CTLQ_DESC_UNUSED(R) \
+ (u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1)
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IECM_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IECM_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IECM_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IECM_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IECM_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct iecm_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IECM_CTLQ_DESC_VF_ID_S 0
+#define IECM_CTLQ_DESC_VF_ID_M (0x7FF << IECM_CTLQ_DESC_VF_ID_S)
+#define IECM_CTLQ_DESC_PF_ID_S 11
+#define IECM_CTLQ_DESC_PF_ID_M (0x1F << IECM_CTLQ_DESC_PF_ID_S)
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IECM_CTLQ_FLAG_DD_S 0
+#define IECM_CTLQ_FLAG_CMP_S 1
+#define IECM_CTLQ_FLAG_ERR_S 2
+#define IECM_CTLQ_FLAG_FTYPE_S 6
+#define IECM_CTLQ_FLAG_RD_S 10
+#define IECM_CTLQ_FLAG_VFC_S 11
+#define IECM_CTLQ_FLAG_BUF_S 12
+#define IECM_CTLQ_FLAG_HOST_ID_S 13
+
+#define IECM_CTLQ_FLAG_DD BIT(IECM_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IECM_CTLQ_FLAG_CMP BIT(IECM_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IECM_CTLQ_FLAG_ERR BIT(IECM_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IECM_CTLQ_FLAG_FTYPE_VM BIT(IECM_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IECM_CTLQ_FLAG_FTYPE_PF BIT(IECM_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IECM_CTLQ_FLAG_RD BIT(IECM_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IECM_CTLQ_FLAG_VFC BIT(IECM_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IECM_CTLQ_FLAG_BUF BIT(IECM_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IECM_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IECM_CTLQ_FLAG_HOST_ID_S)
+
+struct iecm_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum iecm_mac_type {
+ IECM_MAC_UNKNOWN = 0,
+ IECM_MAC_PF,
+ IECM_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct iecm_mac_info {
+ enum iecm_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IECM_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum iecm_bus_type {
+ iecm_bus_type_unknown = 0,
+ iecm_bus_type_pci,
+ iecm_bus_type_pcix,
+ iecm_bus_type_pci_express,
+ iecm_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum iecm_bus_speed {
+ iecm_bus_speed_unknown = 0,
+ iecm_bus_speed_33 = 33,
+ iecm_bus_speed_66 = 66,
+ iecm_bus_speed_100 = 100,
+ iecm_bus_speed_120 = 120,
+ iecm_bus_speed_133 = 133,
+ iecm_bus_speed_2500 = 2500,
+ iecm_bus_speed_5000 = 5000,
+ iecm_bus_speed_8000 = 8000,
+ iecm_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum iecm_bus_width {
+ iecm_bus_width_unknown = 0,
+ iecm_bus_width_pcie_x1 = 1,
+ iecm_bus_width_pcie_x2 = 2,
+ iecm_bus_width_pcie_x4 = 4,
+ iecm_bus_width_pcie_x8 = 8,
+ iecm_bus_width_32 = 32,
+ iecm_bus_width_64 = 64,
+ iecm_bus_width_reserved
+};
+
+/* Bus parameters */
+struct iecm_bus_info {
+ enum iecm_bus_speed speed;
+ enum iecm_bus_width width;
+ enum iecm_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct iecm_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct iecm_hw {
+ u8 *hw_addr;
+ u64 hw_addr_len;
+ void *back;
+
+ /* control queue - send and receive */
+ struct iecm_ctlq_info *asq;
+ struct iecm_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct iecm_mac_info mac;
+ struct iecm_bus_info bus;
+ struct iecm_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, iecm_ctlq_info) cq_list_head;
+};
+
+int iecm_ctlq_alloc_ring_res(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq);
+
+void iecm_ctlq_dealloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *iecm_alloc_dma_mem(struct iecm_hw *hw, struct iecm_dma_mem *mem,
+ u64 size);
+void iecm_free_dma_mem(struct iecm_hw *hw, struct iecm_dma_mem *mem);
+#endif /* _IECM_CONTROLQ_H_ */
diff --git a/drivers/net/idpf/base/iecm_controlq_api.h b/drivers/net/idpf/base/iecm_controlq_api.h
new file mode 100644
index 0000000000..27511ffd51
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq_api.h
@@ -0,0 +1,227 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_CONTROLQ_API_H_
+#define _IECM_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "iecm_mem.h"
+#else /* !__KERNEL__ */
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IECM_SUCCESS 0
+#define IECM_ERR_PARAM -53 /* -EBADR */
+#define IECM_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IECM_ERR_NOT_READY -16 /* -EBUSY */
+#define IECM_ERR_BAD_PTR -14 /* -EFAULT */
+#define IECM_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IECM_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IECM_ERR_FW_API_VER -13 /* -EACCESS */
+#define IECM_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IECM_ERR_CFG -22 /* -EINVAL */
+#define IECM_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IECM_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IECM_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IECM_ERR_IN_USE -114 /* -EALREADY */
+#define IECM_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IECM_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IECM_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IECM_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IECM_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IECM_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IECM_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct iecm_hw;
+
+/* Used for queue init, response and events */
+enum iecm_ctlq_type {
+ IECM_CTLQ_TYPE_MAILBOX_TX = 0,
+ IECM_CTLQ_TYPE_MAILBOX_RX = 1,
+ IECM_CTLQ_TYPE_CONFIG_TX = 2,
+ IECM_CTLQ_TYPE_CONFIG_RX = 3,
+ IECM_CTLQ_TYPE_EVENT_RX = 4,
+ IECM_CTLQ_TYPE_RDMA_TX = 5,
+ IECM_CTLQ_TYPE_RDMA_RX = 6,
+ IECM_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct iecm_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct iecm_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IECM_VMVF_TYPE_VF 0
+#define IECM_VMVF_TYPE_VM 1
+#define IECM_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IECM_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IECM_DIRECT_CTX_SIZE 16
+#define IECM_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IECM_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IECM_INDIRECT_CTX_SIZE];
+ struct iecm_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct iecm_ctlq_create_info {
+ enum iecm_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct iecm_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct iecm_ctlq_info {
+ LIST_ENTRY_TYPE(iecm_ctlq_info) cq_list;
+
+ enum iecm_ctlq_type cq_type;
+ int q_id;
+ iecm_lock cq_lock; /* queue lock
+ * iecm_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct iecm_dma_mem desc_ring; /* descriptor ring memory
+ * iecm_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct iecm_dma_mem **rx_buff;
+ struct iecm_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct iecm_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum iecm_mbx_opc {
+ /* iecm_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ iecm_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* iecm_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ iecm_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* iecm_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ iecm_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* iecm_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ iecm_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+int iecm_ctlq_init(struct iecm_hw *hw, u8 num_q,
+ struct iecm_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int iecm_ctlq_add(struct iecm_hw *hw,
+ struct iecm_ctlq_create_info *qinfo,
+ struct iecm_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void iecm_ctlq_remove(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+int iecm_ctlq_send(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq,
+ u16 num_q_msg,
+ struct iecm_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+int iecm_ctlq_recv(struct iecm_ctlq_info *cq, u16 *num_q_msg,
+ struct iecm_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+int iecm_ctlq_clean_sq(struct iecm_ctlq_info *cq, u16 *clean_count,
+ struct iecm_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+int iecm_ctlq_post_rx_buffs(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq,
+ u16 *buff_count,
+ struct iecm_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+int iecm_ctlq_deinit(struct iecm_hw *hw);
+
+#endif /* _IECM_CONTROLQ_API_H_ */
diff --git a/drivers/net/idpf/base/iecm_controlq_setup.c b/drivers/net/idpf/base/iecm_controlq_setup.c
new file mode 100644
index 0000000000..eb6cf7651d
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "iecm_controlq.h"
+
+
+/**
+ * iecm_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+iecm_ctlq_alloc_desc_ring(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct iecm_ctlq_desc);
+
+ cq->desc_ring.va = iecm_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IECM_ERR_NO_MEMORY;
+
+ return IECM_SUCCESS;
+}
+
+/**
+ * iecm_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int iecm_ctlq_alloc_bufs(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ return IECM_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct iecm_dma_mem **)
+ iecm_calloc(hw, cq->ring_size,
+ sizeof(struct iecm_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IECM_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct iecm_dma_mem *bi;
+ int num = 1; /* number of iecm_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct iecm_dma_mem *)iecm_calloc(hw, num,
+ sizeof(struct iecm_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = iecm_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IECM_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ iecm_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ }
+ iecm_free(hw, cq->bi.rx_buff);
+
+ return IECM_ERR_NO_MEMORY;
+}
+
+/**
+ * iecm_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void iecm_ctlq_free_desc_ring(struct iecm_hw *hw,
+ struct iecm_ctlq_info *cq)
+{
+ iecm_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * iecm_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void iecm_ctlq_free_bufs(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ iecm_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ iecm_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ iecm_free(hw, bi);
+}
+
+/**
+ * iecm_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void iecm_ctlq_dealloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ iecm_ctlq_free_bufs(hw, cq);
+ iecm_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * iecm_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int iecm_ctlq_alloc_ring_res(struct iecm_hw *hw, struct iecm_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IECM_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = iecm_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = iecm_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto iecm_init_cq_free_ring;
+
+ /* success! */
+ return IECM_SUCCESS;
+
+iecm_init_cq_free_ring:
+ iecm_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/net/idpf/base/iecm_devids.h b/drivers/net/idpf/base/iecm_devids.h
new file mode 100644
index 0000000000..aa58982db2
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_DEVIDS_H_
+#define _IECM_DEVIDS_H_
+
+/* Vendor ID */
+#define IECM_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IECM_DEV_ID_PF 0x1452
+#define IECM_DEV_ID_CPF 0x1453
+
+
+
+
+#endif /* _IECM_DEVIDS_H_ */
diff --git a/drivers/net/idpf/base/iecm_lan_pf_regs.h b/drivers/net/idpf/base/iecm_lan_pf_regs.h
new file mode 100644
index 0000000000..c6c460dab0
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_PF_REGS_H_
+#define _IECM_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/net/idpf/base/iecm_lan_txrx.h b/drivers/net/idpf/base/iecm_lan_txrx.h
new file mode 100644
index 0000000000..3e5320975d
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_TXRX_H_
+#define _IECM_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "iecm_osdep.h"
+#endif
+
+enum iecm_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IECM_HASH_INVALID = 0,
+ IECM_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IECM_HASH_NONF_MULTICAST_IPV4_UDP,
+ IECM_HASH_NONF_IPV4_UDP,
+ IECM_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IECM_HASH_NONF_IPV4_TCP,
+ IECM_HASH_NONF_IPV4_SCTP,
+ IECM_HASH_NONF_IPV4_OTHER,
+ IECM_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IECM_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IECM_HASH_NONF_MULTICAST_IPV6_UDP,
+ IECM_HASH_NONF_IPV6_UDP,
+ IECM_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IECM_HASH_NONF_IPV6_TCP,
+ IECM_HASH_NONF_IPV6_SCTP,
+ IECM_HASH_NONF_IPV6_OTHER,
+ IECM_HASH_FRAG_IPV6,
+ IECM_HASH_NONF_RSVD47,
+ IECM_HASH_NONF_FCOE_OX,
+ IECM_HASH_NONF_FCOE_RX,
+ IECM_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IECM_HASH_L2_PAYLOAD = 63,
+ IECM_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IECM_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IECM_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IECM_HASH_FRAG_IPV4) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IECM_HASH_FRAG_IPV6) | \
+ BIT_ULL(IECM_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap belwo comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IECM_DEFAULT_RSS_HASH_EXPANDED (IECM_DEFAULT_RSS_HASH | \
+ BIT_ULL(IECM_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IECM_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IECM_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IECM_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For iecm_splitq_base_tx_compl_desc */
+#define IECM_TXD_COMPLQ_GEN_S 15
+#define IECM_TXD_COMPLQ_GEN_M BIT_ULL(IECM_TXD_COMPLQ_GEN_S)
+#define IECM_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IECM_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IECM_TXD_COMPLQ_COMPL_TYPE_S)
+#define IECM_TXD_COMPLQ_QID_S 0
+#define IECM_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IECM_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IECM_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IECM_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IECM_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IECM_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IECM_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IECM_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IECM_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IECM_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IECM_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IECM_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IECM_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IECM_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IECM_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_UDP_TUNNELING BIT_ULL(IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_GRE_TUNNELING (0x2ULL << IECM_TXD_CTX_QW0_TUNN_NATT_S)
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IECM_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IECM_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IECM_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IECM_TXD_CTX_QW1_MSS_S 50
+#define IECM_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IECM_TXD_CTX_QW1_MSS_S)
+#define IECM_TXD_CTX_QW1_TSO_LEN_S 30
+#define IECM_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IECM_TXD_CTX_QW1_TSO_LEN_S)
+#define IECM_TXD_CTX_QW1_CMD_S 4
+#define IECM_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IECM_TXD_CTX_QW1_CMD_S)
+#define IECM_TXD_CTX_QW1_DTYPE_S 0
+#define IECM_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IECM_TXD_CTX_QW1_DTYPE_S)
+#define IECM_TXD_QW1_L2TAG1_S 48
+#define IECM_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IECM_TXD_QW1_L2TAG1_S)
+#define IECM_TXD_QW1_TX_BUF_SZ_S 34
+#define IECM_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IECM_TXD_QW1_TX_BUF_SZ_S)
+#define IECM_TXD_QW1_OFFSET_S 16
+#define IECM_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IECM_TXD_QW1_OFFSET_S)
+#define IECM_TXD_QW1_CMD_S 4
+#define IECM_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IECM_TXD_QW1_CMD_S)
+#define IECM_TXD_QW1_DTYPE_S 0
+#define IECM_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IECM_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IECM_TXD_COMPLT_ITR_FLUSH 0
+#define IECM_TXD_COMPLT_RULE_MISS 1
+#define IECM_TXD_COMPLT_RS 2
+#define IECM_TXD_COMPLT_REINJECTED 3
+#define IECM_TXD_COMPLT_RE 4
+#define IECM_TXD_COMPLT_SW_MARKER 5
+
+enum iecm_tx_desc_dtype_value {
+ IECM_TX_DESC_DTYPE_DATA = 0,
+ IECM_TX_DESC_DTYPE_CTX = 1,
+ IECM_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IECM_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IECM_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IECM_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IECM_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IECM_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IECM_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IECM_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IECM_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IECM_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum iecm_tx_ctx_desc_cmd_bits {
+ IECM_TX_CTX_DESC_TSO = 0x01,
+ IECM_TX_CTX_DESC_TSYN = 0x02,
+ IECM_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IECM_TX_CTX_DESC_RSVD = 0x08,
+ IECM_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IECM_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IECM_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IECM_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IECM_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IECM_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IECM_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum iecm_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IECM_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IECM_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IECM_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IECM_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IECM_TX_DESC_LEN_MACLEN_S)
+#define IECM_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IECM_TX_DESC_LEN_IPLEN_S)
+#define IECM_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IECM_TX_DESC_LEN_L4_LEN_S)
+#define IECM_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IECM_TX_DESC_LEN_L4_LEN_S)
+
+enum iecm_tx_base_desc_cmd_bits {
+ IECM_TX_DESC_CMD_EOP = 0x0001,
+ IECM_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IECM_TX_DESC_CMD_ICRC = 0x0004,
+ IECM_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IECM_TX_DESC_CMD_RSVD1 = 0x0010,
+ IECM_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IECM_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IECM_TX_DESC_CMD_RSVD2 = 0x0080,
+ IECM_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IECM_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IECM_TX_DESC_CMD_RSVD3 = 0x0400,
+ IECM_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct iecm_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct iecm_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct iecm_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum iecm_tx_flex_desc_cmd_bits {
+ IECM_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IECM_TX_FLEX_DESC_CMD_RS = 0x02,
+ IECM_TX_FLEX_DESC_CMD_RE = 0x04,
+ IECM_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IECM_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IECM_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IECM_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IECM_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct iecm_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IECM_FLEX_TXD_QW1_DTYPE_S 0
+#define IECM_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IECM_FLEX_TXD_QW1_DTYPE_S)
+#define IECM_FLEX_TXD_QW1_CMD_S 5
+#define IECM_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IECM_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IECM_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct iecm_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IECM_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IECM_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IECM_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IECM_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IECM_TXD_FLEX_FLOW_RXR 0x4000
+#define IECM_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum iecm_tx_flex_ctx_desc_cmd_bits {
+ IECM_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IECM_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IECM_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IECM_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct iecm_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IECM_TXD_FLEX_CTX_TLEN_M 0x1FFFF
+#define IECM_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IECM_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union iecm_flex_tx_ctx_desc {
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct iecm_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct iecm_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IECM_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IECM_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IECM_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IECM_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IECM_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct iecm_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IECM_TXD_FLEX_CTX_TLEN_S 0
+#define IECM_TXD_FLEX_CTX_TLEN_M 0x1FFFF
+#define IECM_TXD_FLEX_CTX_FNUM_S 18
+#define IECM_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IECM_TXD_FLEX_CTX_HOST_S 29
+#define IECM_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IECM_TXD_FLEX_CTX_MSS_RT_0 0
+#define IECM_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IECM_TXD_FLEX_CTX_FTYPE_S 14
+#define IECM_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IECM_TXD_FLEX_CTX_FTYPE_S)
+#define IECM_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IECM_TXD_FLEX_CTX_FTYPE_S)
+#define IECM_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IECM_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IECM_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IECM_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IECM_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IECM_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_PASID_VALID_S)
+#define IECM_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IECM_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_TPH_S)
+#define IECM_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IECM_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IECM_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IECM_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IECM_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IECM_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IECM_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IECM_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IECM_LAN_TXRX_H_ */
diff --git a/drivers/net/idpf/base/iecm_lan_vf_regs.h b/drivers/net/idpf/base/iecm_lan_vf_regs.h
new file mode 100644
index 0000000000..1ba1a8dea6
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_LAN_VF_REGS_H_
+#define _IECM_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/net/idpf/base/iecm_prototype.h b/drivers/net/idpf/base/iecm_prototype.h
new file mode 100644
index 0000000000..cd3ee8dcbc
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_PROTOTYPE_H_
+#define _IECM_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "iecm_osdep.h"
+#include "iecm_controlq.h"
+#include "iecm_type.h"
+#include "iecm_alloc.h"
+#include "iecm_devids.h"
+#include "iecm_controlq_api.h"
+#include "iecm_lan_pf_regs.h"
+#include "iecm_lan_vf_regs.h"
+#include "iecm_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int iecm_init_hw(struct iecm_hw *hw, struct iecm_ctlq_size ctlq_size);
+int iecm_deinit_hw(struct iecm_hw *hw);
+
+int iecm_clean_arq_element(struct iecm_hw *hw,
+ struct iecm_arq_event_info *e,
+ u16 *events_pending);
+bool iecm_asq_done(struct iecm_hw *hw);
+bool iecm_check_asq_alive(struct iecm_hw *hw);
+
+int iecm_get_rss_lut(struct iecm_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int iecm_set_rss_lut(struct iecm_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int iecm_get_rss_key(struct iecm_hw *hw, u16 seid,
+ struct iecm_get_set_rss_key_data *key);
+int iecm_set_rss_key(struct iecm_hw *hw, u16 seid,
+ struct iecm_get_set_rss_key_data *key);
+
+int iecm_set_mac_type(struct iecm_hw *hw);
+
+int iecm_reset(struct iecm_hw *hw);
+int iecm_send_msg_to_cp(struct iecm_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IECM_PROTOTYPE_H_ */
diff --git a/drivers/net/idpf/base/iecm_type.h b/drivers/net/idpf/base/iecm_type.h
new file mode 100644
index 0000000000..fdde9c6e61
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_TYPE_H_
+#define _IECM_TYPE_H_
+
+#include "iecm_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p) (_p);
+#define UNREFERENCED_2PARAMETER(_p, _q) (_p); (_q);
+#define UNREFERENCED_3PARAMETER(_p, _q, _r) (_p); (_q); (_r);
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s) (_p); (_q); (_r); (_s);
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t) (_p); (_q); (_r); (_s); (_t);
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct iecm_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct iecm_hw_port_stats {
+ /* eth stats collected by the port */
+ struct iecm_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct iecm_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct iecm_arq_event_info {
+ struct iecm_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct iecm_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct iecm_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct iecm_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IECM_TYPE_H_ */
diff --git a/drivers/net/idpf/base/meson.build b/drivers/net/idpf/base/meson.build
new file mode 100644
index 0000000000..1ad9a87d9d
--- /dev/null
+++ b/drivers/net/idpf/base/meson.build
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = [
+ 'iecm_common.c',
+ 'iecm_controlq.c',
+ 'iecm_controlq_setup.c',
+]
+
+error_cflags = ['-Wno-unused-value',
+ '-Wno-unused-but-set-variable',
+ '-Wno-unused-variable',
+ '-Wno-unused-parameter',
+]
+
+c_args = cflags
+
+foreach flag: error_cflags
+ if cc.has_argument(flag)
+ c_args += flag
+ endif
+endforeach
+
+base_lib = static_library('idpf_base', sources,
+ dependencies: static_rte_eal,
+ c_args: c_args)
+base_objs = base_lib.extract_all_objects()
\ No newline at end of file
diff --git a/drivers/net/idpf/base/siov_regs.h b/drivers/net/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..bb7b2daac0
--- /dev/null
+++ b/drivers/net/idpf/base/siov_regs.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000 /* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000)) /* 2k Rx buffer queues */
+
+#define VDEV_QTX_TAIL_START 0x1100000 /* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000 /* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000)) /* 2k Tx completion queues */
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000) /* Begin at offset of 33MB + 4k to accomdate CTL01 register */
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/net/idpf/base/virtchnl.h b/drivers/net/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..b5d0d5ffd3
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl.h
@@ -0,0 +1,2743 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ **/
+ int count; /* the proto layers must < VIRTCHNL_MAX_NUM_PROTO_HDRS */
+ struct virtchnl_proto_hdr proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/net/idpf/base/virtchnl2.h b/drivers/net/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..566afe075f
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl2.h
@@ -0,0 +1,1411 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+
+#define VIRTCHNL2_MAX_NUM_PROTO_HDRS 32
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids upto 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is trasmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+struct virtchnl2_proto_hdr {
+ /* see VIRTCHNL2_PROTO_HDR_TYPE definitions */
+ __le32 type;
+ __le32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /*
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL2_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_proto_hdr);
+
+struct virtchnl2_proto_hdrs {
+ u8 tunnel_level;
+ /*
+ * specify where protocol header start from.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ __le32 count; /* the proto layers must < VIRTCHNL2_MAX_NUM_PROTO_HDRS */
+ struct virtchnl2_proto_hdr proto_hdr[VIRTCHNL2_MAX_NUM_PROTO_HDRS];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(2312, virtchnl2_proto_hdrs);
+
+struct virtchnl2_rss_cfg {
+ struct virtchnl2_proto_hdrs proto_hdrs;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ u8 reserved[128];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(2444, virtchnl2_rss_cfg);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to HMA to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * HMA responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to HMA */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting HMA to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to HMA */
+ __le16 mbx_vec_id;
+ /* HMA populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* HMA populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to HMA */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to HMA to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * HMA responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(__le16);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/net/idpf/base/virtchnl2_lan_desc.h b/drivers/net/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..2243b17673
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,603 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ | 16| 0|
+ ----------------------------------------------------------------
+ | RSV | Buffer ID |
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ | 0|
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/net/idpf/base/virtchnl_inline_ipsec.h b/drivers/net/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..902f63bd51
--- /dev/null
+++ b/drivers/net/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length caculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 01/14] net/idpf/base: introduce base code Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 13:20 ` Andrew Rybchenko
2022-10-12 8:07 ` Wu, Wenjun1
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
` (13 subsequent siblings)
15 siblings, 2 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Add PMD logs.
Add some MACRO definations and small functions which are specific
for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/base/iecm_osdep.h | 365 +++++++++++++++++++++++++++++
drivers/net/idpf/idpf_logs.h | 38 +++
2 files changed, 403 insertions(+)
create mode 100644 drivers/net/idpf/base/iecm_osdep.h
create mode 100644 drivers/net/idpf/idpf_logs.h
diff --git a/drivers/net/idpf/base/iecm_osdep.h b/drivers/net/idpf/base/iecm_osdep.h
new file mode 100644
index 0000000000..60e21fbc1b
--- /dev/null
+++ b/drivers/net/idpf/base/iecm_osdep.h
@@ -0,0 +1,365 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IECM_OSDEP_H_
+#define _IECM_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#include "../idpf_logs.h"
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum iecm_status iecm_status;
+typedef struct iecm_lock iecm_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0]))
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+#define DEBUGOUT(S) PMD_DRV_LOG_RAW(DEBUG, S)
+#define DEBUGOUT2(S, A...) PMD_DRV_LOG_RAW(DEBUG, S, ##A)
+#define DEBUGFUNC(F) PMD_DRV_LOG_RAW(DEBUG, F)
+
+#define iecm_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "iecm %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define iecm_info(hw, fmt, args...) iecm_debug(hw, IECM_DBG_ALL, fmt, ##args)
+#define iecm_warn(hw, fmt, args...) iecm_debug(hw, IECM_DBG_ALL, fmt, ##args)
+#define iecm_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct iecm_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ iecm_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define iecm_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF iecm_snprintf
+#endif
+
+#define IECM_PCI_REG(reg) rte_read32(reg)
+#define IECM_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IECM_PCI_REG64(reg) rte_read64(reg)
+#define IECM_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define iecm_wmb() rte_io_wmb()
+#define iecm_rmb() rte_io_rmb()
+#define iecm_mb() rte_io_mb()
+
+static inline uint32_t iecm_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IECM_PCI_REG(addr));
+}
+
+static inline uint64_t iecm_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IECM_PCI_REG64(addr));
+}
+
+#define IECM_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IECM_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IECM_READ_REG(hw, reg) iecm_read_addr(IECM_PCI_REG_ADDR((hw), (reg)))
+#define IECM_WRITE_REG(hw, reg, value) \
+ IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) iecm_read_addr(IECM_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) iecm_read_addr64(IECM_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct iecm_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct iecm_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define iecm_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define iecm_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define iecm_free(h, m) rte_free(m)
+
+#define iecm_memset(a, b, c, d) memset((a), (b), (c))
+#define iecm_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define iecm_memdup(a, b, c, d) rte_memcpy(iecm_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct iecm_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+iecm_init_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+iecm_acquire_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+iecm_release_lock(struct iecm_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+iecm_destroy_lock(__attribute__((unused)) struct iecm_lock *sp)
+{
+}
+
+struct iecm_hw;
+
+static inline void *
+iecm_alloc_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
+ struct iecm_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "iecm_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+iecm_free_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
+ struct iecm_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+iecm_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+iecm_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define iecm_usec_delay(x) rte_delay_us(x)
+#define iecm_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IECM_DBG_TRACE
+#define IECM_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IECM_INTEL_VENDOR_ID
+#define IECM_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IECM_OSDEP_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..906aae8463
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,38 @@
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 03/14] net/idpf: add support for device initialization
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 01/14] net/idpf/base: introduce base code Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-09-21 5:41 ` Xing, Beilei
` (3 more replies)
2022-09-05 10:58 ` [PATCH v2 04/14] net/idpf: add support for queue operations Junfeng Guo
` (12 subsequent siblings)
15 siblings, 4 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Xiaoyun Li
Support device init and the following dev ops:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 810 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 229 ++++++++++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++++++
drivers/net/idpf/meson.build | 18 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
6 files changed, 1556 insertions(+)
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..f0452de09e
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,810 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+uint64_t idpf_timestamp_dynflag;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->next_vport_idx;
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static uint16_t
+idpf_get_next_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb,
+ uint16_t cur_vport_idx)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ if (cur_vport_idx < max_vport_nb && !vports[cur_vport_idx + 1]) {
+ vport_idx = cur_vport_idx + 1;
+ return vport_idx;
+ }
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->next_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ adapter->vports[idx] = vport;
+ adapter->next_vport_idx = idpf_get_next_vport_idx(adapter->vports,
+ adapter->max_vport_nb, idx);
+ if (adapter->next_vport_idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Failed to get next vport id");
+ return -1;
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created", adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct iecm_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IECM_READ_REG(hw, PFGEN_CTRL);
+ IECM_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct iecm_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IECM_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct iecm_hw *hw)
+{
+ struct iecm_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IECM_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IECM_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct iecm_ctlq_info *ctlq;
+ int ret;
+
+ ret = iecm_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct iecm_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type == IECM_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type == IECM_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ iecm_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+ adapter->next_vport_idx = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ iecm_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->next_vport_idx]);
+err:
+ return ret;
+}
+
+static int
+idpf_dev_uninit(struct rte_eth_dev *dev)
+{
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return -EPERM;
+
+ idpf_dev_close(dev);
+
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IECM_INTEL_VENDOR_ID, IECM_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse devargs");
+ return retval;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to creat vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ int i;
+
+ iecm_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ char name[RTE_ETH_NAME_MAX_LEN];
+ struct rte_eth_dev *eth_dev;
+ int i;
+
+ for (i = 0; i < IDPF_MAX_VPORT_NUM; i++) {
+ if (adapter->cur_vports & BIT(i)) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name, i);
+ eth_dev = rte_eth_dev_allocated(name);
+ if (eth_dev)
+ rte_eth_dev_destroy(eth_dev, idpf_dev_uninit);
+ }
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..e60ba73ec1
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,229 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "base/iecm_osdep.h"
+#include "base/iecm_type.h"
+#include "base/iecm_devids.h"
+#include "base/iecm_lan_txrx.h"
+#include "base/iecm_lan_pf_regs.h"
+#include "base/virtchnl.h"
+#include "base/virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct iecm_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t next_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_caps(struct idpf_adapter *adapter);
+int idpf_create_vport(struct rte_eth_dev *dev);
+int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..4708fb0666
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "base/iecm_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct iecm_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct iecm_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = iecm_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ iecm_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct iecm_ctlq_msg *ctlq_msg;
+ struct iecm_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct iecm_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct iecm_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct iecm_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct iecm_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ iecm_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = iecm_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = iecm_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ iecm_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_ipf(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct iecm_hw *hw = &adapter->hw;
+ struct iecm_ctlq_msg ctlq_msg;
+ struct iecm_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = iecm_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IECM_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from ipf carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from ipf",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = iecm_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ iecm_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_ipf(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->next_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..7d776d3a15
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+subdir('base')
+objs = [base_objs]
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
+
+includes += include_directories('base')
\ No newline at end of file
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..b7da224860
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
\ No newline at end of file
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index e35652fe63..8faf8120c2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 04/14] net/idpf: add support for queue operations
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 13:47 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 05/14] net/idpf: add support for device information get Junfeng Guo
` (11 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 49 ++
drivers/net/idpf/idpf_ethdev.h | 7 +
drivers/net/idpf/idpf_rxtx.c | 1270 ++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 190 +++++
drivers/net/idpf/idpf_vchnl.c | 509 +++++++++++++
drivers/net/idpf/meson.build | 1 +
6 files changed, 2026 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f0452de09e..0349ec6e9d 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,6 +10,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_VPORT "vport"
@@ -33,6 +34,14 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static int
@@ -209,6 +218,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -218,6 +260,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -226,6 +273,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index e60ba73ec1..4a98e7739b 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -222,6 +222,13 @@ int idpf_check_api_version(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
int idpf_create_vport(struct rte_eth_dev *dev);
int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_config_rxqs(struct idpf_vport *vport);
+int idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_config_txqs(struct idpf_vport *vport);
+int idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..fe044a80c9
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,1270 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct iecm_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct iecm_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct iecm_base_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1 =
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct iecm_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ bufq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ bufq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t qid;
+ uint16_t len;
+ uint64_t offloads;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_rx_queue *rxq;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t len;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct iecm_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct iecm_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct iecm_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct iecm_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct iecm_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct iecm_base_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct iecm_base_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
+
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+#endif
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IECM_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IECM_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..d930bb75ff
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,190 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "base/iecm_osdep.h"
+#include "base/iecm_type.h"
+#include "base/iecm_devids.h"
+#include "base/iecm_lan_txrx.h"
+#include "base/iecm_lan_pf_regs.h"
+#include "base/virtchnl.h"
+#include "base/virtchnl2.h"
+#include "base/virtchnl2_lan_desc.h"
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 256
+#define IDPF_MAX_TSO_MSS 9668
+#define IDPF_TSO_MAX_SEG UINT8_MAX
+#define IDPF_TX_MAX_MTU_SEG 8
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t crc_len; /* 0 if CRC stripped, 4 otherwise */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct iecm_base_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct iecm_flex_tx_sched_desc *desc_ring;
+ struct iecm_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
+
+#endif /* _IDPF_RXTX_H_ */
+
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4708fb0666..cbc2a5ba12 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "base/iecm_prototype.h"
@@ -469,6 +470,514 @@ idpf_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i, j;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+static int
+idpf_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 7d776d3a15..338a39e391 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,6 +12,7 @@ objs = [base_objs]
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 05/14] net/idpf: add support for device information get
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 04/14] net/idpf: add support for queue operations Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 13:53 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 06/14] net/idpf: add support for packet type get Junfeng Guo
` (10 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 80 ++++++++++++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0349ec6e9d..8852b7ce22 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -28,6 +28,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
@@ -42,8 +44,86 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_SCATTER |
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT |
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT |
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 06/14] net/idpf: add support for packet type get
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 05/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 13:58 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 07/14] net/idpf: add support for link status update Junfeng Guo
` (9 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 +++++++++++++++++++++++++++++++++
5 files changed, 269 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 8852b7ce22..0ed2c99dd2 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -647,6 +648,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 4a98e7739b..1833c6e57d 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -219,6 +219,7 @@ _atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
int idpf_create_vport(struct rte_eth_dev *dev);
int idpf_destroy_vport(struct idpf_vport *vport);
@@ -230,6 +231,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index fe044a80c9..1c5c4688cc 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index d930bb75ff..f2947a8492 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -46,6 +46,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -186,5 +190,7 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index cbc2a5ba12..00855f153e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_get_caps(struct idpf_adapter *adapter)
{
@@ -1002,3 +1211,29 @@ idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+int
+idpf_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
+
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 07/14] net/idpf: add support for link status update
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 06/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (8 subsequent siblings)
15 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Add dev ops link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 21 +++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 2 ++
2 files changed, 23 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0ed2c99dd2..2cbf2f0e19 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -31,6 +31,26 @@ static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
@@ -46,6 +66,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 1833c6e57d..e3a2a98c5b 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,8 @@ _atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
}
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_check_api_version(struct idpf_adapter *adapter);
int idpf_get_pkt_type(struct idpf_adapter *adapter);
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 07/14] net/idpf: add support for link status update Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 14:02 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 09/14] net/idpf: add support for RSS Junfeng Guo
` (7 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Xiaoyun Li
Add basic RX & TX support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 47 +-
drivers/net/idpf/idpf_rxtx.c | 896 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 28 ++
3 files changed, 966 insertions(+), 5 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 2cbf2f0e19..6310745684 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -12,6 +12,8 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
@@ -20,6 +22,8 @@ bool adapter_list_init;
uint64_t idpf_timestamp_dynflag;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -367,6 +371,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -393,6 +400,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -519,6 +528,26 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", "
+ "value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -537,7 +566,20 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
adapter);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+bail:
rte_kvargs_free(kvlist);
return ret;
}
@@ -763,8 +805,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 1c5c4688cc..54d83a7c61 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -990,10 +990,6 @@ idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
rxd->pkt_addr = dma_addr;
rxd->hdr_addr = 0;
-#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
- rxd->rsvd1 = 0;
- rxd->rsvd2 = 0;
-#endif
rxq->sw_ring[i] = mbuf;
}
@@ -1287,3 +1283,895 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_ERR0_QW1 \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_ERR0_QW1) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = rte_le_to_cpu_16(rx_desc->hash1) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_HASH3_S);
+ }
+
+ return flags;
+}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->nb_rx_hold;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[nb_refill];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+
+ if (nb_refill <= rx_bufq->rx_free_thresh)
+ return;
+
+ if (nb_refill >= nb_desc)
+ nb_refill = nb_desc - 1;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (delta < nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (int i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (int i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IECM_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ pkt_len -= rxq->crc_len;
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ rxm->ol_flags |= pkt_flags;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct iecm_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct iecm_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_GEN_M) >> IECM_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_COMPL_TYPE_M) >> IECM_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IECM_TXD_COMPLQ_QID_M) >> IECM_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IECM_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IECM_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union iecm_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IECM_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IECM_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IECM_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IECM_TXD_FLEX_CTX_MSS_RT_M);
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct iecm_flex_tx_sched_desc *txr;
+ volatile struct iecm_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ingnore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union iecm_flex_tx_ctx_desc *ctx_desc =
+ (volatile union iecm_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IECM_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IECM_TXD_FLEX_FLOW_CMD_CS_EN;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IECM_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M) -
+ rxq->crc_len;
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+
+ volatile struct iecm_base_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ if ((txd[desc_to_clean_to].qw1 &
+ rte_cpu_to_le_64(IECM_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1 = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ * support IP -> L4 and IP -> IP -> L4
+ */
+static inline uint64_t
+idpf_set_tso_ctx(struct rte_mbuf *mbuf, union idpf_tx_offload tx_offload)
+{
+ uint64_t ctx_desc = 0;
+ uint32_t cd_cmd, hdr_len, cd_tso_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return ctx_desc;
+ }
+
+ hdr_len = tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len;
+
+ cd_cmd = IECM_TX_CTX_DESC_TSO;
+ cd_tso_len = mbuf->pkt_len - hdr_len;
+ ctx_desc |= ((uint64_t)cd_cmd << IECM_TXD_CTX_QW1_CMD_S) |
+ ((uint64_t)cd_tso_len << IECM_TXD_CTX_QW1_TSO_LEN_S) |
+ ((uint64_t)mbuf->tso_segsz << IECM_TXD_CTX_QW1_MSS_S);
+
+ return ctx_desc;
+}
+
+/* Construct the tx flags */
+static inline uint64_t
+idpf_build_ctob(uint32_t td_cmd, uint32_t td_offset, unsigned int size)
+{
+ return rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)td_cmd << IECM_TXD_QW1_CMD_S) |
+ ((uint64_t)td_offset <<
+ IECM_TXD_QW1_OFFSET_S) |
+ ((uint64_t)size <<
+ IECM_TXD_QW1_TX_BUF_SZ_S));
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct iecm_base_tx_desc *txd;
+ volatile struct iecm_base_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint32_t td_offset;
+ uint64_t ol_flags;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t nb_ctx;
+ uint32_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+ td_offset = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ /* According to datasheet, the bit2 is reserved and must be
+ * set to 1.
+ */
+ td_cmd |= 0x04;
+
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union iecm_flex_tx_ctx_desc *ctx_txd =
+ (volatile union iecm_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1 = idpf_build_ctob(td_cmd, td_offset, slen);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IECM_TX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IECM_TX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1 |=
+ rte_cpu_to_le_64(((uint64_t)td_cmd) <<
+ IECM_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IECM_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
+ if (!m->pkt_len) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ return;
+ } else {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ return;
+ }
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f2947a8492..3ccf9efe50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -43,6 +43,22 @@
#define IDPF_TSO_MAX_SEG UINT8_MAX
#define IDPF_TX_MAX_MTU_SEG 8
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
@@ -187,8 +203,20 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 09/14] net/idpf: add support for RSS
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 14:10 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 10/14] net/idpf: add support for mtu configuration Junfeng Guo
` (6 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 118 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 17 +++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++++
3 files changed, 229 insertions(+), 2 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6310745684..b1e2ca21ca 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -87,6 +87,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -319,9 +320,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
{
- return 0;
+ int ret;
+
+ ret = idpf_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IECM_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -420,6 +528,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index e3a2a98c5b..a37582d7b9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -53,6 +53,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -225,6 +239,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_get_caps(struct idpf_adapter *adapter);
int idpf_create_vport(struct rte_eth_dev *dev);
int idpf_destroy_vport(struct idpf_vport *vport);
+int idpf_set_rss_key(struct idpf_vport *vport);
+int idpf_set_rss_lut(struct idpf_vport *vport);
+int idpf_set_rss_hash(struct idpf_vport *vport);
int idpf_config_rxqs(struct idpf_vport *vport);
int idpf_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 00855f153e..6f9b24ab62 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_config_rxqs(struct idpf_vport *vport)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 10/14] net/idpf: add support for mtu configuration
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 09/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 14:12 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 11/14] net/idpf: add support for hw statistics Junfeng Guo
` (5 subsequent siblings)
15 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index b1e2ca21ca..6b3e7eff89 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -34,6 +34,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
int
idpf_dev_link_update(struct rte_eth_dev *dev,
@@ -71,6 +72,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -151,6 +153,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -474,6 +488,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 11/14] net/idpf: add support for hw statistics
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 10/14] net/idpf: add support for mtu configuration Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (4 subsequent siblings)
15 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Mingxia Liu
Add hardware packets/bytes statisticsi query and reset.
Signed-off-by: Mingxia Liu <mingxia.liu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 78 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_vchnl.c | 27 ++++++++++++
3 files changed, 107 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6b3e7eff89..9fee7a783f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,9 @@ static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
+static int idpf_dev_stats_get(struct rte_eth_dev *dev,
+ struct rte_eth_stats *stats);
+static int idpf_dev_stats_reset(struct rte_eth_dev *dev);
int
idpf_dev_link_update(struct rte_eth_dev *dev,
@@ -73,6 +76,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
.mtu_set = idpf_dev_mtu_set,
+ .stats_get = idpf_dev_stats_get,
+ .stats_reset = idpf_dev_stats_reset,
};
static int
@@ -165,6 +170,75 @@ idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
return 0;
}
+static void
+idpf_stat_update(uint64_t *offset, uint64_t *stat)
+{
+ *stat = *stat - *offset;
+}
+
+
+static void
+idpf_update_stats(struct virtchnl2_vport_stats *oes, struct virtchnl2_vport_stats *nes)
+{
+ idpf_stat_update(&oes->rx_bytes, &nes->rx_bytes);
+ idpf_stat_update(&oes->rx_unicast, &nes->rx_unicast);
+ idpf_stat_update(&oes->rx_multicast, &nes->rx_multicast);
+ idpf_stat_update(&oes->rx_broadcast, &nes->rx_broadcast);
+ idpf_stat_update(&oes->rx_discards, &nes->rx_discards);
+ idpf_stat_update(&oes->tx_bytes, &nes->tx_bytes);
+ idpf_stat_update(&oes->tx_unicast, &nes->tx_unicast);
+ idpf_stat_update(&oes->tx_multicast, &nes->tx_multicast);
+ idpf_stat_update(&oes->tx_broadcast, &nes->tx_broadcast);
+ idpf_stat_update(&oes->tx_errors, &nes->tx_errors);
+ idpf_stat_update(&oes->tx_discards, &nes->tx_discards);
+}
+
+static int
+idpf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct virtchnl2_vport_stats *pstats = NULL;
+ int ret;
+
+ ret = idpf_query_stats(vport, &pstats);
+ if (ret == 0) {
+ uint8_t crc_stats_len = (dev->data->dev_conf.rxmode.offloads &
+ RTE_ETH_RX_OFFLOAD_KEEP_CRC) ? 0 :
+ RTE_ETHER_CRC_LEN;
+ idpf_update_stats(&vport->eth_stats_offset, pstats);
+ stats->ipackets = pstats->rx_unicast + pstats->rx_multicast +
+ pstats->rx_broadcast - pstats->rx_discards;
+ stats->opackets = pstats->tx_broadcast + pstats->tx_multicast +
+ pstats->tx_unicast;
+ stats->imissed = pstats->rx_discards;
+ stats->oerrors = pstats->tx_errors + pstats->tx_discards;
+ stats->ibytes = pstats->rx_bytes;
+ stats->ibytes -= stats->ipackets * crc_stats_len;
+ stats->obytes = pstats->tx_bytes;
+ } else {
+ PMD_DRV_LOG(ERR, "Get statistics failed");
+ }
+ return ret;
+}
+
+
+static int
+idpf_dev_stats_reset(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct virtchnl2_vport_stats *pstats = NULL;
+ int ret;
+
+ ret = idpf_query_stats(vport, &pstats);
+ if (ret != 0)
+ return ret;
+
+ /* set stats offset base on current values */
+ vport->eth_stats_offset = *pstats;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -508,6 +582,10 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_vport;
}
+ if (idpf_dev_stats_reset(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to reset stats");
+ goto err_vport;
+ }
return 0;
err_vport:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index a37582d7b9..2657b75c95 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -250,6 +250,8 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_query_stats(struct idpf_vport *vport,
+ struct virtchnl2_vport_stats **pstats);
int idpf_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 6f9b24ab62..49572dc83c 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1307,6 +1307,33 @@ idpf_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_query_stats(struct idpf_vport *vport,
+ struct virtchnl2_vport_stats **pstats)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = VIRTCHNL2_OP_GET_STATS;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_STATS");
+ *pstats = NULL;
+ return err;
+ }
+ *pstats = (struct virtchnl2_vport_stats *)args.out_buffer;
+ return 0;
+}
+
int
idpf_query_ptype_info(struct idpf_adapter *adapter)
{
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 12/14] net/idpf: add support for write back based on ITR expire
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 11/14] net/idpf: add support for hw statistics Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
` (3 subsequent siblings)
15 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang, junfeng.guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_vchnl.c | 108 ++++++++++++++++++++++++++++++
3 files changed, 227 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 9fee7a783f..b3ca4e3326 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -520,6 +520,87 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct iecm_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start = vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start = vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IECM_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x", dynctl_val);
+ itrn_val = IECM_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are writen back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IECM_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IECM_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -557,6 +638,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -569,6 +654,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -609,6 +711,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -633,6 +741,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -982,6 +1096,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vport_nb = 0;
adapter->next_vport_idx = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 2657b75c95..f96867f3d5 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -252,6 +252,9 @@ int idpf_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_query_stats(struct idpf_vport *vport,
struct virtchnl2_vport_stats **pstats);
+int idpf_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_dealloc_vectors(struct idpf_vport *vport);
int idpf_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 49572dc83c..97fcfb574b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1133,6 +1133,114 @@ idpf_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+
+int
+idpf_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 14:20 ` Andrew Rybchenko
2022-10-10 8:06 ` Wu, Wenjun1
2022-09-05 10:58 ` [PATCH v2 14/14] net/idpf: add support for timestamp offload Junfeng Guo
` (2 subsequent siblings)
15 siblings, 2 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 137 ++++
drivers/net/idpf/idpf_rxtx.h | 11 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 917 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 89 +++
drivers/net/idpf/meson.build | 31 +-
6 files changed, 1189 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index f96867f3d5..a32d5758ac 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -179,6 +179,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 54d83a7c61..e31d202646 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2146,15 +2148,110 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+#ifdef RTE_ARCH_X86
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
return;
}
@@ -2164,12 +2261,52 @@ void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_tx_queue *txq;
+ int i;
+
+#ifdef RTE_ARCH_X86
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3ccf9efe50..decd0a98c2 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -122,6 +122,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -207,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..70eb5e1e7e
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,917 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 2 features are supported in RX path,
+ * 1, checksum offload
+ * 2, VLAN/QINQ stripping
+ * 3, RSS hash
+ * 4, packet type analysis
+ * 5, flow director ID report
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+#else
+ const __m512i permute_idx = _mm512_set_epi64(7, 3, 6, 2, 5, 1, 4, 0);
+#endif
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+#else
+ /* permute leaves desc 4-7 addresses in header address slots 0-3
+ * but these are ignored by driver since header split not
+ * enabled.
+ */
+ const __m512i desc0_3 = _mm512_permutexvar_epi64(permute_idx,
+ iova_addrs);
+ const __m512i desc4_7 = _mm512_bsrli_epi128(desc0_3, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_7);
+#endif
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* constants used in processing loop */
+ const __m512i crc_adjust =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 2nd descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 3rd descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0, /* ignore pkt_type field */
+ /* 4th descriptor */
+ 0, /* ignore non-length fields */
+ -rxq->crc_len, /* sub crc on data_len */
+ -rxq->crc_len, /* sub crc on pkt_len */
+ 0 /* ignore pkt_type field */
+ );
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* octet 4~7, 32bits rss */
+ 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the above crc and shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+ mb4_7 = _mm512_add_epi32(mb4_7, crc_adjust);
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+ mb0_3 = _mm512_add_epi32(mb0_3, crc_adjust);
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value in vlan0.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1 &
+ rte_cpu_to_le_64(IECM_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline void
+idpf_vtx1(volatile struct iecm_base_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)flags << IECM_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IECM_TXD_QW1_TX_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct iecm_base_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IECM_TX_DESC_DTYPE_DATA |
+ ((uint64_t)flags << IECM_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IECM_TXD_QW1_TX_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct iecm_base_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ /* bit2 is reserved and must be set to 1 according to Spec */
+ uint64_t flags = IECM_TX_DESC_CMD_EOP | 0x04;
+ uint64_t rs = IECM_TX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1 |=
+ rte_cpu_to_le_64(((uint64_t)IECM_TX_DESC_CMD_RS) <<
+ IECM_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IECM_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..336d4c8b25
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ /* Currently, vector path doesn't support timestamp. */
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 338a39e391..ee4bb94a97 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -16,4 +16,33 @@ sources = files(
'idpf_vchnl.c',
)
-includes += include_directories('base')
\ No newline at end of file
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
+
+includes += include_directories('base')
+
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v2 14/14] net/idpf: add support for timestamp offload
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-09-05 10:58 ` Junfeng Guo
2022-10-03 14:22 ` Andrew Rybchenko
2022-10-10 7:56 ` Wu, Wenjun1
2022-10-03 13:31 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Andrew Rybchenko
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
15 siblings, 2 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-09-05 10:58 UTC (permalink / raw)
To: qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, junfeng.guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 79 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 89 +++++++++++++++++++++++++++++++++-
3 files changed, 170 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index a32d5758ac..968e0e3cbf 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -184,6 +184,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e31d202646..b0037eca08 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -965,6 +967,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(
+ &idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -992,6 +1012,10 @@ idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
rxd->pkt_addr = dma_addr;
rxd->hdr_addr = 0;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+#endif
rxq->sw_ring[i] = mbuf;
}
@@ -1057,6 +1081,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1441,6 +1472,12 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ uint64_t ts_ns;
+ struct iecm_hw *hw = &rxq->adapter->hw;
+ struct idpf_adapter *ad = rxq->adapter;
+#endif
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1451,6 +1488,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
(volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+#endif
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1507,6 +1549,19 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ if (idpf_timestamp_dynflag > 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(hw, ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+#endif
rxm->ol_flags |= pkt_flags;
rx_pkts[nb_rx++] = rxm;
@@ -1778,6 +1833,10 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
nb_hold = 0;
rxq = rx_queue;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ uint64_t ts_ns;
+#endif
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1785,6 +1844,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ struct iecm_hw *hw = &rxq->adapter->hw;
+ struct idpf_adapter *ad = rxq->adapter;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+#endif
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1841,6 +1907,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
+ if (idpf_timestamp_dynflag > 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(hw, ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxdp->flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+#endif
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index decd0a98c2..6fcb441143 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,6 +15,41 @@
#include "base/virtchnl2_lan_desc.h"
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -66,6 +101,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,5 +268,55 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
-#endif /* _IDPF_RXTX_H_ */
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+idpf_tstamp_convert_32b_64b(struct iecm_hw *hw, struct idpf_adapter *ad,
+ uint32_t flag, uint32_t in_timestamp)
+{
+/* TODO: timestamp for ACC */
+#ifdef RTE_ARCH_ARM64
+ return 0;
+#endif /* RTE_ARCH_ARM64 */
+
+#ifdef RTE_ARCH_X86_64
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#endif /* RTE_ARCH_X86_64 */
+}
+#endif /* _IDPF_RXTX_H_ */
--
2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 03/14] net/idpf: add support for device initialization
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-09-21 5:41 ` Xing, Beilei
2022-09-21 6:04 ` Xing, Beilei
` (2 subsequent siblings)
3 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-09-21 5:41 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing; +Cc: dev, Wang, Xiao W, Li, Xiaoyun
> -----Original Message-----
> From: Guo, Junfeng <junfeng.guo@intel.com>
> Sent: Monday, September 5, 2022 6:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Guo, Junfeng
> <junfeng.guo@intel.com>; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: [PATCH v2 03/14] net/idpf: add support for device initialization
>
> Support device init and the following dev ops:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 810
> +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_ethdev.h |
> 229 ++++++++++ drivers/net/idpf/idpf_vchnl.c | 495
> ++++++++++++++++++++
> drivers/net/idpf/meson.build | 18 +
> drivers/net/idpf/version.map | 3 +
> drivers/net/meson.build | 1 +
> 6 files changed, 1556 insertions(+)
> create mode 100644 drivers/net/idpf/idpf_ethdev.c create mode 100644
> drivers/net/idpf/idpf_ethdev.h create mode 100644
> drivers/net/idpf/idpf_vchnl.c create mode 100644
> drivers/net/idpf/meson.build create mode 100644
> drivers/net/idpf/version.map
>
<...>
> +static int
> +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter
> +*adapter) {
> + struct iecm_hw *hw = &adapter->hw;
> + int ret = 0;
> +
> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
> + hw->back = adapter;
> + hw->vendor_id = pci_dev->id.vendor_id;
> + hw->device_id = pci_dev->id.device_id;
> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
> +
> + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
> +
> + idpf_reset_pf(hw);
> + ret = idpf_check_pf_reset_done(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
> + goto err;
> + }
> +
> + ret = idpf_init_mbx(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
> + goto err;
> + }
> +
> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
> IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->mbx_resp) {
> + PMD_INIT_LOG(ERR, "Failed to allocate
> idpf_adapter_mbx_resp memory");
> + goto err_mbx;
> + }
> +
> + if (idpf_check_api_version(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to check api version");
> + goto err_api;
> + }
> +
> + adapter->caps = rte_zmalloc("idpf_caps",
> + sizeof(struct virtchnl2_get_capabilities), 0);
> + if (!adapter->caps) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
> + goto err_api;
> + }
> +
> + if (idpf_get_caps(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
> + goto err_caps;
> + }
> +
> + adapter->max_vport_nb = adapter->caps->max_vports;
> +
> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_req_info),
> + 0);
> + if (!adapter->vport_req_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info
> memory");
> + goto err_caps;
> + }
> +
> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter-
> >vport_recv_info),
> + 0);
> + if (!adapter->vport_recv_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info
> memory");
> + goto err_vport_recv_info;
> + }
> +
> + adapter->vports = rte_zmalloc("vports",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vports),
> + 0);
> + if (!adapter->vports) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
> + goto err_vports;
> + }
> +
> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_rx_queues)) /
> + sizeof(struct virtchnl2_rxq_info);
> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_tx_queues)) /
> + sizeof(struct virtchnl2_txq_info);
> +
> + adapter->cur_vports = 0;
> + adapter->cur_vport_nb = 0;
> + adapter->next_vport_idx = 0;
> +
> + return ret;
> +
> +err_vports:
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
Need to free adapter->vport_recv_info here but not adapter->vports.
> +err_vport_recv_info:
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> +err_caps:
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +err_api:
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +err_mbx:
> + iecm_ctlq_deinit(hw);
> +err:
> + return -1;
> +}
> +
> +
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 03/14] net/idpf: add support for device initialization
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
2022-09-21 5:41 ` Xing, Beilei
@ 2022-09-21 6:04 ` Xing, Beilei
2022-10-03 13:44 ` Andrew Rybchenko
2022-10-10 7:48 ` Wu, Wenjun1
3 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-09-21 6:04 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing; +Cc: dev, Wang, Xiao W, Li, Xiaoyun
> +static void
> +idpf_adapter_rel(struct idpf_adapter *adapter) {
> + struct iecm_hw *hw = &adapter->hw;
> + int i;
> +
> + iecm_ctlq_deinit(hw);
> +
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +
> + if (adapter->vport_req_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_req_info[i]);
> + adapter->vport_req_info[i] = NULL;
> + }
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> + }
> +
> + if (adapter->vport_recv_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_recv_info[i]);
> + adapter->vport_recv_info[i] = NULL;
> + }
Also need to free adapter->vport_recv_info here.
> + }
> +
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +}
> +
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 01/14] net/idpf/base: introduce base code
2022-09-05 10:58 ` [PATCH v2 01/14] net/idpf/base: introduce base code Junfeng Guo
@ 2022-10-03 13:20 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:20 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> Introduce base code for IDPF (Infrastructure Data Path Function) PMD.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
I'm OK to import base driver in one patch, but it should not be
the first patch with dead code which is not actually built
just after the patch.
The first patch should add a driver stub which is built.
Subsequent patches should add base driver and functionality
to the ethdev driver itself.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation
2022-09-05 10:58 ` [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation Junfeng Guo
@ 2022-10-03 13:20 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-10-12 8:07 ` Wu, Wenjun1
1 sibling, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:20 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> Add PMD logs.
> Add some MACRO definations and small functions which are specific
> for DPDK.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
The patch adds dead code as well. Added headers are used
nowhere.
One more question is what's the point to have two things in one
patch? Typically such descriptions suggest that it could be
separate patches in a right order.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 00/14] add support for idpf PMD in DPDK
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (13 preceding siblings ...)
2022-09-05 10:58 ` [PATCH v2 14/14] net/idpf: add support for timestamp offload Junfeng Guo
@ 2022-10-03 13:31 ` Andrew Rybchenko
2022-10-03 14:36 ` Andrew Rybchenko
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
15 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:31 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> This patchset introduced the idpf (Infrastructure Data Path Function)
> PMD [*EXPERIMENTAL*] in DPDK for Intel Device ID of 0x1452.
>
> v2:
> fixed some coding style issues and did some refactors.
The patch series should:
- add entry in MAINTAINERS
- add driver documentation to doc/guides/nics/idpf.rst
- advertise the driver in release notes
- when features are added it should be added to
doc/guides/nics/features/idpf.ini
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 03/14] net/idpf: add support for device initialization
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
2022-09-21 5:41 ` Xing, Beilei
2022-09-21 6:04 ` Xing, Beilei
@ 2022-10-03 13:44 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-10-10 7:48 ` Wu, Wenjun1
3 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:44 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Xiaoyun Li
On 9/5/22 13:58, Junfeng Guo wrote:
> Support device init and the following dev ops:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
A bit strange set from my point of view since you can't start
without setup queues. Each patch should be testable.
It should be no dead code. It should be possible to stop after
the patch, build, run, for example, testpmd and probe,
configure, start, stop, close, unplug (depeneding on stage in
the patch series).
The patch is really big. It definitely worse to split
configre/close and start/stop (and reorder to have start
after queues setup supported).
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 810 +++++++++++++++++++++++++++++++++
> drivers/net/idpf/idpf_ethdev.h | 229 ++++++++++
> drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++++++
> drivers/net/idpf/meson.build | 18 +
> drivers/net/idpf/version.map | 3 +
> drivers/net/meson.build | 1 +
> 6 files changed, 1556 insertions(+)
> create mode 100644 drivers/net/idpf/idpf_ethdev.c
> create mode 100644 drivers/net/idpf/idpf_ethdev.h
> create mode 100644 drivers/net/idpf/idpf_vchnl.c
> create mode 100644 drivers/net/idpf/meson.build
> create mode 100644 drivers/net/idpf/version.map
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> new file mode 100644
> index 0000000000..f0452de09e
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -0,0 +1,810 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#define IDPF_VPORT "vport"
> +
> +struct idpf_adapter_list adapter_list;
> +bool adapter_list_init;
> +
> +uint64_t idpf_timestamp_dynflag;
> +
> +static const char * const idpf_valid_args[] = {
> + IDPF_VPORT,
> + NULL
> +};
> +
> +static int idpf_dev_configure(struct rte_eth_dev *dev);
> +static int idpf_dev_start(struct rte_eth_dev *dev);
> +static int idpf_dev_stop(struct rte_eth_dev *dev);
> +static int idpf_dev_close(struct rte_eth_dev *dev);
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_start = idpf_dev_start,
> + .dev_stop = idpf_dev_stop,
> + .dev_close = idpf_dev_close,
> +};
> +
> +static int
> +idpf_init_vport_req_info(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_create_vport *vport_info;
> + uint16_t idx = adapter->next_vport_idx;
> +
> + if (!adapter->vport_req_info[idx]) {
DPDK coding style requires explicit comparison vs NULL [1].
It is not applicable to base driver, but PMD itself should
follow it. There is really huge number of places to fix.
It is applicable to comparision integers vs 0 as well.
[1] https://doc.dpdk.org/guides/contributing/coding_style.html#null-pointers
[snip]
> +static int
> +idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
> +{
I'm really confusing that the configure as empty.
It must validate configuration provided via
rte_eth_dev_configure() and reject if something is not
supported if ethdev does not the job.
> + return 0;
> +}
[snip]
> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> new file mode 100644
> index 0000000000..7d776d3a15
> --- /dev/null
> +++ b/drivers/net/idpf/meson.build
> @@ -0,0 +1,18 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 Intel Corporation
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
> +endif
> +
> +subdir('base')
> +objs = [base_objs]
> +
> +sources = files(
> + 'idpf_ethdev.c',
> + 'idpf_vchnl.c',
TAB vs spaces? Or is it just mail glitch?
> +)
> +
> +includes += include_directories('base')
> \ No newline at end of file
> diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
> new file mode 100644
> index 0000000000..b7da224860
> --- /dev/null
> +++ b/drivers/net/idpf/version.map
> @@ -0,0 +1,3 @@
> +DPDK_22 {
> + local: *;
> +};
> \ No newline at end of file
It must be an empty line at the end of file.
> diff --git a/drivers/net/meson.build b/drivers/net/meson.build
> index e35652fe63..8faf8120c2 100644
> --- a/drivers/net/meson.build
> +++ b/drivers/net/meson.build
> @@ -28,6 +28,7 @@ drivers = [
> 'i40e',
> 'iavf',
> 'ice',
> + 'idpf',
TAB vs spaces? Or is it just mail glitch?
> 'igc',
> 'ionic',
> 'ipn3ke',
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 04/14] net/idpf: add support for queue operations
2022-09-05 10:58 ` [PATCH v2 04/14] net/idpf: add support for queue operations Junfeng Guo
@ 2022-10-03 13:47 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:47 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Xiaoyun Li
On 9/5/22 13:58, Junfeng Guo wrote:
> Add support for queue operations:
> - rx_queue_start
> - rx_queue_stop
> - tx_queue_start
> - tx_queue_stop
> - rx_queue_setup
> - rx_queue_release
> - tx_queue_setup
> - tx_queue_release
The patch is huge. Please, factor out deferred queues
start/stop (may be even separate for Rx and Tx) into
separate patch.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 05/14] net/idpf: add support for device information get
2022-09-05 10:58 ` [PATCH v2 05/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-03 13:53 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:53 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> Add dev ops dev_infos_get.
The hook may be called on very earlier stages of the driver
life. So, I'd recommend to move the patch even before
configure/clsoe. User may request the dev_info in just
probed state.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 80 ++++++++++++++++++++++++++++++++++
> 1 file changed, 80 insertions(+)
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index 0349ec6e9d..8852b7ce22 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -28,6 +28,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
> static int idpf_dev_start(struct rte_eth_dev *dev);
> static int idpf_dev_stop(struct rte_eth_dev *dev);
> static int idpf_dev_close(struct rte_eth_dev *dev);
> +static int idpf_dev_info_get(struct rte_eth_dev *dev,
> + struct rte_eth_dev_info *dev_info);
>
> static const struct eth_dev_ops idpf_eth_dev_ops = {
> .dev_configure = idpf_dev_configure,
> @@ -42,8 +44,86 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
> .rx_queue_release = idpf_dev_rx_queue_release,
> .tx_queue_setup = idpf_tx_queue_setup,
> .tx_queue_release = idpf_dev_tx_queue_release,
> + .dev_infos_get = idpf_dev_info_get,
> };
>
> +static int
> +idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + dev_info->max_rx_queues = adapter->caps->max_rx_q;
> + dev_info->max_tx_queues = adapter->caps->max_tx_q;
> + dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
> + dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
> +
> + dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
> + dev_info->min_mtu = RTE_ETHER_MIN_MTU;
> +
> + dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> + dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
All above feature must be advertised as soon as it is really
supported by the driver. Do you really support runtime queues
setup at this point? In fact it is a separate feature and its
better to have dedicated patch to support it which updates
reported device capabilties.
> + dev_info->rx_offload_capa =
> + RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
All offloads must be reported as soon at it is actually
supported. So, the patch which adds support for an offload
must update the device info callback to advirtise it.
> + RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
> + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_SCATTER |
> + RTE_ETH_RX_OFFLOAD_VLAN_FILTER |
> + RTE_ETH_RX_OFFLOAD_RSS_HASH |
> + RTE_ETH_RX_OFFLOAD_TIMESTAMP;
> +
> + dev_info->tx_offload_capa =
> + RTE_ETH_TX_OFFLOAD_VLAN_INSERT |
> + RTE_ETH_TX_OFFLOAD_QINQ_INSERT |
> + RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
> + RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
> + RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
> + RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
> + RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
> + RTE_ETH_TX_OFFLOAD_TCP_TSO |
> + RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
> + RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
> + RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
> + RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO |
> + RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
> + RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
> +
> + dev_info->default_rxconf = (struct rte_eth_rxconf) {
> + .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
> + .rx_drop_en = 0,
> + .offloads = 0,
> + };
> +
> + dev_info->default_txconf = (struct rte_eth_txconf) {
> + .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
> + .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
> + .offloads = 0,
> + };
> +
> + dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
> + .nb_max = IDPF_MAX_RING_DESC,
> + .nb_min = IDPF_MIN_RING_DESC,
> + .nb_align = IDPF_ALIGN_RING_DESC,
> + };
> +
> + dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
> + .nb_max = IDPF_MAX_RING_DESC,
> + .nb_min = IDPF_MIN_RING_DESC,
> + .nb_align = IDPF_ALIGN_RING_DESC,
> + };
> +
> + dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
> + dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
Is it really an intent to recommend maximum burst side
by default?
> + dev_info->default_rxportconf.nb_queues = 1;
> + dev_info->default_txportconf.nb_queues = 1;
> +
> + return 0;
> +}
> +
> static int
> idpf_init_vport_req_info(struct rte_eth_dev *dev)
> {
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 06/14] net/idpf: add support for packet type get
2022-09-05 10:58 ` [PATCH v2 06/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-03 13:58 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 13:58 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Wenjun Wu
On 9/5/22 13:58, Junfeng Guo wrote:
> Add dev ops dev_supported_ptypes_get.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> index fe044a80c9..1c5c4688cc 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -8,6 +8,25 @@
> #include "idpf_ethdev.h"
> #include "idpf_rxtx.h"
>
> +const uint32_t *
> +idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
> +{
> + static const uint32_t ptypes[] = {
> + RTE_PTYPE_L2_ETHER,
> + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
> + RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
> + RTE_PTYPE_L4_FRAG,
> + RTE_PTYPE_L4_NONFRAG,
> + RTE_PTYPE_L4_UDP,
> + RTE_PTYPE_L4_TCP,
> + RTE_PTYPE_L4_SCTP,
> + RTE_PTYPE_L4_ICMP,
> + RTE_PTYPE_UNKNOWN
> + };
How am I supported to verify that these packet types are really
recognized by the driver? The patch should include the part
of Rx burst callback which does recognition and fill in
corresponding data in mbuf.
> +
> + return ptypes;
> +}
> +
> static inline int
> check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
> {
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-09-05 10:58 ` [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-03 14:02 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:02 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Xiaoyun Li
On 9/5/22 13:58, Junfeng Guo wrote:
> Add basic RX & TX support in split queue mode and single queue mode.
RX -> Rx, TX -> Tx
> Split queue mode is selected by default.
What is split queue mode? Where is it defined/described?
The patch is very big again. Please, start from really basic Rx
and Tx functionality and then add offloads one by one.
Otherwise it is really hard to review everything at once.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 09/14] net/idpf: add support for RSS
2022-09-05 10:58 ` [PATCH v2 09/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-03 14:10 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:10 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> Add RSS support.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 118 ++++++++++++++++++++++++++++++++-
> drivers/net/idpf/idpf_ethdev.h | 17 +++++
> drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++++
> 3 files changed, 229 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index 6310745684..b1e2ca21ca 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -87,6 +87,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
> dev_info->min_mtu = RTE_ETHER_MIN_MTU;
>
> + dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
> dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
> @@ -319,9 +320,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
> }
>
> static int
> -idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
> +idpf_config_rss(struct idpf_vport *vport)
> {
> - return 0;
> + int ret;
> +
> + ret = idpf_set_rss_key(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS key");
> + return ret;
> + }
> +
> + ret = idpf_set_rss_lut(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
> + return ret;
> + }
> +
> + ret = idpf_set_rss_hash(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_init_rss(struct idpf_vport *vport)
> +{
> + struct rte_eth_rss_conf *rss_conf;
> + uint16_t i, nb_q, lut_size;
> + int ret = 0;
> +
> + rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
> + nb_q = vport->dev_data->nb_rx_queues;
> +
> + vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
> + vport->rss_key_size, 0);
> + if (!vport->rss_key) {
> + PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
> + ret = -ENOMEM;
> + goto err_key;
> + }
> +
> + lut_size = vport->rss_lut_size;
> + vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
> + sizeof(uint32_t) * lut_size, 0);
> + if (!vport->rss_lut) {
> + PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
> + ret = -ENOMEM;
> + goto err_lut;
> + }
> +
> + if (!rss_conf->rss_key) {
> + for (i = 0; i < vport->rss_key_size; i++)
> + vport->rss_key[i] = (uint8_t)rte_rand();
IMHO it is a bad idea. Random key could result in very
bad distribution of the traffic.
> + } else {
> + rte_memcpy(vport->rss_key, rss_conf->rss_key,
> + RTE_MIN(rss_conf->rss_key_len,
> + vport->rss_key_size));
It looks like rss_key_len from rss_conf is saved nowhere.
How do you know which bits of the RSS key are really valid/
initialized.
> + }
> +
> + for (i = 0; i < lut_size; i++)
> + vport->rss_lut[i] = i % nb_q;
> +
> + vport->rss_hf = IECM_DEFAULT_RSS_HASH_EXPANDED;
Hm, what about rss_conf->rss_hf?
> +
> + ret = idpf_config_rss(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS");
> + goto err_cfg;
> + }
> +
> + return ret;
> +
> +err_cfg:
> + rte_free(vport->rss_lut);
> + vport->rss_lut = NULL;
> +err_lut:
> + rte_free(vport->rss_key);
> + vport->rss_key = NULL;
> +err_key:
> + return ret;
> +}
> +
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + int ret = 0;
> +
> + if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
> + dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
> + PMD_INIT_LOG(ERR, "Invalid queue number.");
> + return -EINVAL;
> + }
Above is checked on ethdev level if you report maximums
correctly.
> +
> + if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
> + dev->data->dev_conf.rxmode.offloads |=
> + RTE_ETH_RX_OFFLOAD_RSS_HASH;
It looks wrong. Offload configuration is a user choice.
> +
> + if (adapter->caps->rss_caps) {
> + ret = idpf_init_rss(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init rss");
> + return ret;
> + }
> + }
> +
> + return ret;
> }
>
> static int
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 10/14] net/idpf: add support for mtu configuration
2022-09-05 10:58 ` [PATCH v2 10/14] net/idpf: add support for mtu configuration Junfeng Guo
@ 2022-10-03 14:12 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:12 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 9/5/22 13:58, Junfeng Guo wrote:
> Add dev ops mtu_set.
It would be useful to mention in the description that
changing MTU is not supported when device is started.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model
2022-09-05 10:58 ` [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-03 14:20 ` Andrew Rybchenko
2022-10-14 9:19 ` Guo, Junfeng
2022-10-10 8:06 ` Wu, Wenjun1
1 sibling, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:20 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Wenjun Wu, Thomas Monjalon
On 9/5/22 13:58, Junfeng Guo wrote:
> Add support of AVX512 vector data path for single queue model.
>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> +static __rte_always_inline void
> +idpf_singleq_rearm(struct idpf_rx_queue *rxq)
> +{
[snip]
> + const __m512i iova_offsets = _mm512_set1_epi64(offsetof
> + (struct rte_mbuf, buf_iova));
> + const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
> +
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
As far as I remember, support for such build time options is
deprecated in DPDK and new options are not accepted.
Cc Thomas to correct me if I'm wrong.
> + /* to shuffle the addresses to correct slots. Values 4-7 will contain
> + * zeros, so use 7 for a zero-value.
> + */
> + const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
> +#else
> + const __m512i permute_idx = _mm512_set_epi64(7, 3, 6, 2, 5, 1, 4, 0);
> +#endif
[snip]
> +#ifdef IDPF_RX_PTYPE_OFFLOAD
Same here.
> + /**
> + * to get packet types, shift 64-bit values down 30 bits
> + * and so ptype is in lower 8-bits in each
> + */
> + const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
> + const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
> + const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
> + const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
> + const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
> + const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
> + const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
> +
> + const __m512i ptype4_7 = _mm512_set_epi32
> + (0, 0, 0, type_table[ptype7],
> + 0, 0, 0, type_table[ptype6],
> + 0, 0, 0, type_table[ptype5],
> + 0, 0, 0, type_table[ptype4]);
> + mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
> +#endif
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 14/14] net/idpf: add support for timestamp offload
2022-09-05 10:58 ` [PATCH v2 14/14] net/idpf: add support for timestamp offload Junfeng Guo
@ 2022-10-03 14:22 ` Andrew Rybchenko
2022-10-14 9:19 ` Guo, Junfeng
2022-10-10 7:56 ` Wu, Wenjun1
1 sibling, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:22 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, xiao.w.wang, Wenjing Qiao
On 9/5/22 13:58, Junfeng Guo wrote:
> Add support for timestamp offload.
>
> Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> +/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
> +static inline uint64_t
> +idpf_tstamp_convert_32b_64b(struct iecm_hw *hw, struct idpf_adapter *ad,
> + uint32_t flag, uint32_t in_timestamp)
> +{
> +/* TODO: timestamp for ACC */
> +#ifdef RTE_ARCH_ARM64
> + return 0;
> +#endif /* RTE_ARCH_ARM64 */
> +
> +#ifdef RTE_ARCH_X86_64
> + const uint64_t mask = 0xFFFFFFFF;
> + uint32_t hi, lo, lo2, delta;
> + uint64_t ns;
> +
> + if (flag) {
> + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
> + PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> + lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
> + /*
> + * On typical system, the delta between lo and lo2 is ~1000ns,
> + * so 10000 seems a large-enough but not overly-big guard band.
> + */
> + if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
> + lo2 = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + else
> + lo2 = lo;
> +
> + if (lo2 < lo) {
> + lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
> + }
> +
> + ad->time_hw = ((uint64_t)hi << 32) | lo;
> + }
> +
> + delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
> + if (delta > (mask / 2)) {
> + delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
> + ns = ad->time_hw - delta;
> + } else {
> + ns = ad->time_hw + delta;
> + }
> +
> + return ns;
> +#endif /* RTE_ARCH_X86_64 */
Conditional compilation is rather strange above.
Will it break build on some architectures?
Non-x86-64 and non-ARM64.
> +}
>
> +#endif /* _IDPF_RXTX_H_ */
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v2 00/14] add support for idpf PMD in DPDK
2022-10-03 13:31 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Andrew Rybchenko
@ 2022-10-03 14:36 ` Andrew Rybchenko
2022-10-18 11:09 ` Guo, Junfeng
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-03 14:36 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, xiao.w.wang
On 10/3/22 16:31, Andrew Rybchenko wrote:
> On 9/5/22 13:58, Junfeng Guo wrote:
>> This patchset introduced the idpf (Infrastructure Data Path Function)
>> PMD [*EXPERIMENTAL*] in DPDK for Intel Device ID of 0x1452.
>>
>> v2:
>> fixed some coding style issues and did some refactors.
>
> The patch series should:
> - add entry in MAINTAINERS
> - add driver documentation to doc/guides/nics/idpf.rst
> - advertise the driver in release notes
> - when features are added it should be added to
> doc/guides/nics/features/idpf.ini
I'd really like to add one more point.
There is a long road for the driver to be accepted since
many changes are required, but it is definitely doable.
More important to get replies on generic questions.
What is it? What is the HW? Is it released?
Where I can read about it?
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 03/14] net/idpf: add support for device initialization
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
` (2 preceding siblings ...)
2022-10-03 13:44 ` Andrew Rybchenko
@ 2022-10-10 7:48 ` Wu, Wenjun1
3 siblings, 0 replies; 376+ messages in thread
From: Wu, Wenjun1 @ 2022-10-10 7:48 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Guo, Junfeng, Li, Xiaoyun
> -----Original Message-----
> From: Junfeng Guo <junfeng.guo@intel.com>
> Sent: Monday, September 5, 2022 6:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Guo, Junfeng
> <junfeng.guo@intel.com>; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: [PATCH v2 03/14] net/idpf: add support for device initialization
>
> Support device init and the following dev ops:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 810
> +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_ethdev.h |
> 229 ++++++++++ drivers/net/idpf/idpf_vchnl.c | 495
> ++++++++++++++++++++
> drivers/net/idpf/meson.build | 18 +
> drivers/net/idpf/version.map | 3 +
> drivers/net/meson.build | 1 +
> 6 files changed, 1556 insertions(+)
> create mode 100644 drivers/net/idpf/idpf_ethdev.c create mode 100644
> drivers/net/idpf/idpf_ethdev.h create mode 100644
> drivers/net/idpf/idpf_vchnl.c create mode 100644
> drivers/net/idpf/meson.build create mode 100644
> drivers/net/idpf/version.map
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> new file mode 100644 index 0000000000..f0452de09e
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -0,0 +1,810 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#define IDPF_VPORT "vport"
> +
> +struct idpf_adapter_list adapter_list;
> +bool adapter_list_init;
> +
> +uint64_t idpf_timestamp_dynflag;
> +
> +static const char * const idpf_valid_args[] = {
> + IDPF_VPORT,
> + NULL
> +};
> +
> +static int idpf_dev_configure(struct rte_eth_dev *dev); static int
> +idpf_dev_start(struct rte_eth_dev *dev); static int
> +idpf_dev_stop(struct rte_eth_dev *dev); static int
> +idpf_dev_close(struct rte_eth_dev *dev);
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_start = idpf_dev_start,
> + .dev_stop = idpf_dev_stop,
> + .dev_close = idpf_dev_close,
> +};
> +
> +static int
> +idpf_init_vport_req_info(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_create_vport *vport_info;
> + uint16_t idx = adapter->next_vport_idx;
> +
> + if (!adapter->vport_req_info[idx]) {
> + adapter->vport_req_info[idx] = rte_zmalloc(NULL,
> + sizeof(struct virtchnl2_create_vport), 0);
> + if (!adapter->vport_req_info[idx]) {
> + PMD_INIT_LOG(ERR, "Failed to allocate
> vport_req_info");
> + return -1;
> + }
> + }
> +
> + vport_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_req_info[idx];
> +
> + vport_info->vport_type =
> rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
> + if (!adapter->txq_model) {
> + vport_info->txq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
> + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
> + vport_info->num_tx_complq =
> + IDPF_DEFAULT_TXQ_NUM *
> IDPF_TX_COMPLQ_PER_GRP;
> + } else {
> + vport_info->txq_model =
> +
> rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
> + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
> + vport_info->num_tx_complq = 0;
> + }
> + if (!adapter->rxq_model) {
> + vport_info->rxq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
> + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
> + vport_info->num_rx_bufq =
> + IDPF_DEFAULT_RXQ_NUM *
> IDPF_RX_BUFQ_PER_GRP;
> + } else {
> + vport_info->rxq_model =
> +
> rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
> + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
> + vport_info->num_rx_bufq = 0;
> + }
> +
> + return 0;
> +}
> +
> +static uint16_t
> +idpf_get_next_vport_idx(struct idpf_vport **vports, uint16_t
> max_vport_nb,
> + uint16_t cur_vport_idx)
> +{
> + uint16_t vport_idx;
> + uint16_t i;
> +
> + if (cur_vport_idx < max_vport_nb && !vports[cur_vport_idx + 1]) {
> + vport_idx = cur_vport_idx + 1;
> + return vport_idx;
> + }
> +
> + for (i = 0; i < max_vport_nb; i++) {
> + if (!vports[i])
> + break;
> + }
> +
> + if (i == max_vport_nb)
> + vport_idx = IDPF_INVALID_VPORT_IDX;
> + else
> + vport_idx = i;
> +
> + return vport_idx;
> +}
> +
> +static uint16_t
> +idpf_parse_devarg_id(char *name)
> +{
> + uint16_t val;
> + char *p;
> +
> + p = strstr(name, "vport_");
> + p += sizeof("vport_") - 1;
> +
> + val = strtoul(p, NULL, 10);
> +
> + return val;
> +}
> +
> +#ifndef IDPF_RSS_KEY_LEN
> +#define IDPF_RSS_KEY_LEN 52
> +#endif
> +
> +static int
> +idpf_init_vport(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t idx = adapter->next_vport_idx;
> + struct virtchnl2_create_vport *vport_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_recv_info[idx];
> + int i;
> +
> + vport->vport_id = vport_info->vport_id;
> + vport->txq_model = vport_info->txq_model;
> + vport->rxq_model = vport_info->rxq_model;
> + vport->num_tx_q = vport_info->num_tx_q;
> + vport->num_tx_complq = vport_info->num_tx_complq;
> + vport->num_rx_q = vport_info->num_rx_q;
> + vport->num_rx_bufq = vport_info->num_rx_bufq;
> + vport->max_mtu = vport_info->max_mtu;
> + rte_memcpy(vport->default_mac_addr,
> + vport_info->default_mac_addr, ETH_ALEN);
> + vport->rss_algorithm = vport_info->rss_algorithm;
> + vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
> + vport_info->rss_key_size);
> + vport->rss_lut_size = vport_info->rss_lut_size;
> + vport->sw_idx = idx;
> +
> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> + if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX) {
> + vport->chunks_info.tx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX) {
> + vport->chunks_info.rx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
> + vport->chunks_info.tx_compl_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_compl_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_compl_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
> + vport->chunks_info.rx_buf_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_buf_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_buf_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + }
> + }
> +
> + adapter->vports[idx] = vport;
> + adapter->next_vport_idx = idpf_get_next_vport_idx(adapter->vports,
> + adapter->max_vport_nb,
> idx);
> + if (adapter->next_vport_idx == IDPF_INVALID_VPORT_IDX) {
> + PMD_INIT_LOG(ERR, "Failed to get next vport id");
> + return -1;
> + }
> +
> + vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
> + vport->dev_data = dev->data;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_configure(__rte_unused struct rte_eth_dev *dev) {
> + return 0;
> +}
> +
> +static int
> +idpf_dev_start(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + vport->stopped = 0;
> +
> + if (idpf_ena_dis_vport(vport, true)) {
> + PMD_DRV_LOG(ERR, "Failed to enable vport");
> + goto err_vport;
> + }
> +
> + return 0;
> +
> +err_vport:
> + return -1;
> +}
> +
> +static int
> +idpf_dev_stop(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + if (vport->stopped == 1)
> + return 0;
> +
> + if (idpf_ena_dis_vport(vport, false))
> + PMD_DRV_LOG(ERR, "disable vport failed");
> +
> + vport->stopped = 1;
> + dev->data->dev_started = 0;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_close(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return 0;
> +
> + idpf_dev_stop(dev);
> + idpf_destroy_vport(vport);
> +
> + adapter->cur_vports &= ~BIT(vport->devarg_id);
> +
> + rte_free(vport);
> + dev->data->dev_private = NULL;
> +
> + return 0;
> +}
> +
> +static int
> +insert_value(struct idpf_adapter *adapter, uint16_t id) {
> + uint16_t i;
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if (adapter->req_vports[i] == id)
> + return 0;
> + }
> +
> + if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + adapter->req_vports[adapter->req_vport_nb] = id;
> + adapter->req_vport_nb++;
> +
> + return 0;
> +}
> +
> +static const char *
> +parse_range(const char *value, struct idpf_adapter *adapter) {
> + uint16_t lo, hi, i;
> + int n = 0;
> + int result;
> + const char *pos = value;
> +
> + result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
> + if (result == 1) {
> + if (lo >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + if (insert_value(adapter, lo))
> + return NULL;
> + } else if (result == 2) {
> + if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + for (i = lo; i <= hi; i++) {
> + if (insert_value(adapter, i))
> + return NULL;
> + }
> + } else {
> + return NULL;
> + }
> +
> + return pos + n;
> +}
> +
> +static int
> +parse_vport(const char *key, const char *value, void *args) {
> + struct idpf_adapter *adapter = (struct idpf_adapter *)args;
> + const char *pos = value;
> + int i;
> +
> + adapter->req_vport_nb = 0;
> +
> + if (*pos == '[')
> + pos++;
> +
> + while (1) {
> + pos = parse_range(pos, adapter);
> + if (pos == NULL) {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for
> key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> + if (*pos != ',')
> + break;
> + pos++;
> + }
> +
> + if (*value == '[' && *pos != ']') {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> +
> + if (adapter->cur_vport_nb + adapter->req_vport_nb >
> + IDPF_MAX_VPORT_NUM) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
> + adapter->cur_vports |= BIT(adapter->req_vports[i]);
> + adapter->cur_vport_nb++;
> + } else {
> + PMD_INIT_LOG(ERR, "Vport %d has been created",
> adapter->req_vports[i]);
> + return -1;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int
> +idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter
> +*adapter) {
> + struct rte_devargs *devargs = pci_dev->device.devargs;
> + struct rte_kvargs *kvlist;
> + int ret;
> +
> + if (!devargs)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
> + if (!kvlist) {
> + PMD_INIT_LOG(ERR, "invalid kvargs key");
> + return -EINVAL;
> + }
> +
> + ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
> + adapter);
> +
> + rte_kvargs_free(kvlist);
> + return ret;
> +}
> +
> +static void
> +idpf_reset_pf(struct iecm_hw *hw)
> +{
> + uint32_t reg;
> +
> + reg = IECM_READ_REG(hw, PFGEN_CTRL);
> + IECM_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR)); }
> +
> +#define IDPF_RESET_WAIT_CNT 100
> +static int
> +idpf_check_pf_reset_done(struct iecm_hw *hw) {
> + uint32_t reg;
> + int i;
> +
> + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
> + reg = IECM_READ_REG(hw, PFGEN_RSTAT);
> + if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
> + return 0;
> + rte_delay_ms(1000);
> + }
> +
> + PMD_INIT_LOG(ERR, "IDPF reset timeout");
> + return -EBUSY;
> +}
> +
> +#define CTLQ_NUM 2
> +static int
> +idpf_init_mbx(struct iecm_hw *hw)
> +{
> + struct iecm_ctlq_create_info ctlq_info[CTLQ_NUM] = {
> + {
> + .type = IECM_CTLQ_TYPE_MAILBOX_TX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ATQH,
> + .tail = PF_FW_ATQT,
> + .len = PF_FW_ATQLEN,
> + .bah = PF_FW_ATQBAH,
> + .bal = PF_FW_ATQBAL,
> + .len_mask = PF_FW_ATQLEN_ATQLEN_M,
> + .len_ena_mask =
> PF_FW_ATQLEN_ATQENABLE_M,
> + .head_mask = PF_FW_ATQH_ATQH_M,
> + }
> + },
> + {
> + .type = IECM_CTLQ_TYPE_MAILBOX_RX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ARQH,
> + .tail = PF_FW_ARQT,
> + .len = PF_FW_ARQLEN,
> + .bah = PF_FW_ARQBAH,
> + .bal = PF_FW_ARQBAL,
> + .len_mask = PF_FW_ARQLEN_ARQLEN_M,
> + .len_ena_mask =
> PF_FW_ARQLEN_ARQENABLE_M,
> + .head_mask = PF_FW_ARQH_ARQH_M,
> + }
> + }
> + };
> + struct iecm_ctlq_info *ctlq;
> + int ret;
> +
> + ret = iecm_ctlq_init(hw, CTLQ_NUM, ctlq_info);
> + if (ret)
> + return ret;
> +
> + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
> + struct iecm_ctlq_info, cq_list) {
> + if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type ==
> IECM_CTLQ_TYPE_MAILBOX_TX)
> + hw->asq = ctlq;
> + if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type ==
> IECM_CTLQ_TYPE_MAILBOX_RX)
> + hw->arq = ctlq;
> + }
> +
> + if (!hw->asq || !hw->arq) {
> + iecm_ctlq_deinit(hw);
> + ret = -ENOENT;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter
> +*adapter) {
> + struct iecm_hw *hw = &adapter->hw;
> + int ret = 0;
> +
> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
> + hw->back = adapter;
> + hw->vendor_id = pci_dev->id.vendor_id;
> + hw->device_id = pci_dev->id.device_id;
> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
> +
> + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
> +
> + idpf_reset_pf(hw);
> + ret = idpf_check_pf_reset_done(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
> + goto err;
> + }
> +
> + ret = idpf_init_mbx(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
> + goto err;
> + }
> +
> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
> IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->mbx_resp) {
> + PMD_INIT_LOG(ERR, "Failed to allocate
> idpf_adapter_mbx_resp memory");
> + goto err_mbx;
> + }
> +
> + if (idpf_check_api_version(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to check api version");
> + goto err_api;
> + }
> +
> + adapter->caps = rte_zmalloc("idpf_caps",
> + sizeof(struct virtchnl2_get_capabilities), 0);
> + if (!adapter->caps) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
> + goto err_api;
> + }
> +
> + if (idpf_get_caps(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
> + goto err_caps;
> + }
> +
> + adapter->max_vport_nb = adapter->caps->max_vports;
> +
> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_req_info),
> + 0);
> + if (!adapter->vport_req_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info
> memory");
> + goto err_caps;
> + }
> +
> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter-
> >vport_recv_info),
> + 0);
> + if (!adapter->vport_recv_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info
> memory");
> + goto err_vport_recv_info;
> + }
> +
> + adapter->vports = rte_zmalloc("vports",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vports),
> + 0);
> + if (!adapter->vports) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
> + goto err_vports;
> + }
> +
> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_rx_queues)) /
> + sizeof(struct virtchnl2_rxq_info);
> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_tx_queues)) /
> + sizeof(struct virtchnl2_txq_info);
> +
> + adapter->cur_vports = 0;
> + adapter->cur_vport_nb = 0;
> + adapter->next_vport_idx = 0;
> +
> + return ret;
> +
> +err_vports:
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +err_vport_recv_info:
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> +err_caps:
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +err_api:
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +err_mbx:
> + iecm_ctlq_deinit(hw);
> +err:
> + return -1;
> +}
> +
> +
> +static int
> +idpf_dev_init(struct rte_eth_dev *dev, void *init_params) {
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = init_params;
> + int ret = 0;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + dev->dev_ops = &idpf_eth_dev_ops;
> + vport->adapter = adapter;
> +
> + /* for secondary processes, we don't initialise any further as primary
> + * has already done this work.
> + */
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return ret;
> +
> + dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> +
> + ret = idpf_init_vport_req_info(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
> + goto err;
> + }
> +
> + ret = idpf_create_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to create vport.");
> + goto err_create_vport;
> + }
> +
> + ret = idpf_init_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vports.");
> + goto err_init_vport;
> + }
> +
> + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN,
> 0);
> + if (dev->data->mac_addrs == NULL) {
> + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
> + ret = -ENOMEM;
> + goto err_init_vport;
> + }
> +
> + rte_ether_addr_copy((struct rte_ether_addr *)vport-
> >default_mac_addr,
> + &dev->data->mac_addrs[0]);
> +
> + return 0;
> +
> +err_init_vport:
> + idpf_destroy_vport(vport);
> +err_create_vport:
> +
> +rte_free(vport->adapter->vport_req_info[vport->adapter->next_vport_idx]
> +);
> +err:
> + return ret;
> +}
> +
> +static int
> +idpf_dev_uninit(struct rte_eth_dev *dev) {
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return -EPERM;
> +
> + idpf_dev_close(dev);
> +
> + return 0;
> +}
> +
> +static const struct rte_pci_id pci_id_idpf_map[] = {
> + { RTE_PCI_DEVICE(IECM_INTEL_VENDOR_ID, IECM_DEV_ID_PF) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +struct idpf_adapter *
> +idpf_find_adapter(struct rte_pci_device *pci_dev) {
> + struct idpf_adapter *adapter;
> +
> + TAILQ_FOREACH(adapter, &adapter_list, next) {
> + if (!strncmp(adapter->name, pci_dev->device.name,
> PCI_PRI_STR_SIZE))
> + return adapter;
> + }
> +
> + return NULL;
> +}
> +
> +static int
> +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev) {
> + struct idpf_adapter *adapter;
> + char name[RTE_ETH_NAME_MAX_LEN];
> + int i, retval;
> +
> + if (!adapter_list_init) {
> + TAILQ_INIT(&adapter_list);
> + adapter_list_init = true;
> + }
> +
> + adapter = idpf_find_adapter(pci_dev);
> + if (!adapter) {
> + adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
> + sizeof(struct idpf_adapter), 0);
> + if (!adapter) {
> + PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
> + return -1;
> + }
> +
> + retval = idpf_adapter_init(pci_dev, adapter);
> + if (retval) {
> + PMD_INIT_LOG(ERR, "Failed to init adapter.");
> + return retval;
> + }
> +
> + TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
> + }
> +
> + retval = idpf_parse_devargs(pci_dev, adapter);
> + if (retval) {
> + PMD_INIT_LOG(ERR, "Failed to parse devargs");
> + return retval;
> + }
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + snprintf(name, sizeof(name), "idpf_%s_vport_%d",
> + pci_dev->device.name,
> + adapter->req_vports[i]);
> + retval = rte_eth_dev_create(&pci_dev->device, name,
> + sizeof(struct idpf_vport),
> + NULL, NULL, idpf_dev_init,
> + adapter);
> + if (retval)
> + PMD_DRV_LOG(ERR, "failed to creat vport %d",
> + adapter->req_vports[i]);
> + }
> +
> + return 0;
> +}
> +
> +static void
> +idpf_adapter_rel(struct idpf_adapter *adapter) {
> + struct iecm_hw *hw = &adapter->hw;
> + int i;
> +
> + iecm_ctlq_deinit(hw);
> +
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +
> + if (adapter->vport_req_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_req_info[i]);
> + adapter->vport_req_info[i] = NULL;
> + }
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> + }
> +
> + if (adapter->vport_recv_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_recv_info[i]);
> + adapter->vport_recv_info[i] = NULL;
> + }
> + }
> +
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +}
> +
> +static int
> +idpf_pci_remove(struct rte_pci_device *pci_dev) {
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
> + char name[RTE_ETH_NAME_MAX_LEN];
> + struct rte_eth_dev *eth_dev;
> + int i;
> +
> + for (i = 0; i < IDPF_MAX_VPORT_NUM; i++) {
> + if (adapter->cur_vports & BIT(i)) {
> + snprintf(name, sizeof(name), "idpf_%s_vport_%d",
> + pci_dev->device.name, i);
> + eth_dev = rte_eth_dev_allocated(name);
> + if (eth_dev)
> + rte_eth_dev_destroy(eth_dev,
> idpf_dev_uninit);
> + }
> + }
> +
> + TAILQ_REMOVE(&adapter_list, adapter, next);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> +
> + return 0;
> +}
> +
> +static struct rte_pci_driver rte_idpf_pmd = {
> + .id_table = pci_id_idpf_map,
> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING |
> RTE_PCI_DRV_PROBE_AGAIN,
> + .probe = idpf_pci_probe,
> + .remove = idpf_pci_remove,
> +};
> +
> +/**
> + * Driver initialization routine.
> + * Invoked once at EAL init time.
> + * Register itself as the [Poll Mode] Driver of PCI devices.
> + */
> +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
> +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic |
> +vfio-pci");
> +
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> new file mode 100644 index 0000000000..e60ba73ec1
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -0,0 +1,229 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_ETHDEV_H_
> +#define _IDPF_ETHDEV_H_
> +
> +#include <stdint.h>
> +#include <rte_mbuf.h>
> +#include <rte_mempool.h>
> +#include <rte_malloc.h>
> +#include <rte_spinlock.h>
> +#include <rte_ethdev.h>
> +#include <rte_kvargs.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +
> +#include "base/iecm_osdep.h"
> +#include "base/iecm_type.h"
> +#include "base/iecm_devids.h"
> +#include "base/iecm_lan_txrx.h"
> +#include "base/iecm_lan_pf_regs.h"
> +#include "base/virtchnl.h"
> +#include "base/virtchnl2.h"
> +
> +#define IDPF_MAX_VPORT_NUM 8
> +
> +#define IDPF_DEFAULT_RXQ_NUM 16
> +#define IDPF_DEFAULT_TXQ_NUM 16
> +
> +#define IDPF_INVALID_VPORT_IDX 0xffff
> +#define IDPF_TXQ_PER_GRP 1
> +#define IDPF_TX_COMPLQ_PER_GRP 1
> +#define IDPF_RXQ_PER_GRP 1
> +#define IDPF_RX_BUFQ_PER_GRP 2
> +
> +#define IDPF_CTLQ_ID -1
> +#define IDPF_CTLQ_LEN 64
> +#define IDPF_DFLT_MBX_BUF_SIZE 4096
> +
> +#define IDPF_DFLT_Q_VEC_NUM 1
> +#define IDPF_DFLT_INTERVAL 16
> +
> +#define IDPF_MAX_NUM_QUEUES 256
> +#define IDPF_MIN_BUF_SIZE 1024
> +#define IDPF_MAX_FRAME_SIZE 9728
> +
> +#define IDPF_NUM_MACADDR_MAX 64
> +
> +#define IDPF_MAX_PKT_TYPE 1024
> +
> +#define IDPF_VLAN_TAG_SIZE 4
> +#define IDPF_ETH_OVERHEAD \
> + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN +
> IDPF_VLAN_TAG_SIZE * 2)
> +
> +#ifndef ETH_ADDR_LEN
> +#define ETH_ADDR_LEN 6
> +#endif
> +
> +#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
> +
> +/* Message type read in virtual channel from PF */ enum idpf_vc_result
> +{
> + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
> + IDPF_MSG_NON, /* Read nothing from admin queue */
> + IDPF_MSG_SYS, /* Read system msg from admin queue */
> + IDPF_MSG_CMD, /* Read async command result */
> +};
> +
> +struct idpf_chunks_info {
> + uint32_t tx_start_qid;
> + uint32_t rx_start_qid;
> + /* Valid only if split queue model */
> + uint32_t tx_compl_start_qid;
> + uint32_t rx_buf_start_qid;
> +
> + uint64_t tx_qtail_start;
> + uint32_t tx_qtail_spacing;
> + uint64_t rx_qtail_start;
> + uint32_t rx_qtail_spacing;
> + uint64_t tx_compl_qtail_start;
> + uint32_t tx_compl_qtail_spacing;
> + uint64_t rx_buf_qtail_start;
> + uint32_t rx_buf_qtail_spacing;
> +};
> +
> +struct idpf_vport {
> + struct idpf_adapter *adapter; /* Backreference to associated adapter
> */
> + uint16_t vport_id;
> + uint32_t txq_model;
> + uint32_t rxq_model;
> + uint16_t num_tx_q;
> + /* valid only if txq_model is split Q */
> + uint16_t num_tx_complq;
> + uint16_t num_rx_q;
> + /* valid only if rxq_model is split Q */
> + uint16_t num_rx_bufq;
> +
> + uint16_t max_mtu;
> + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
> +
> + enum virtchnl_rss_algorithm rss_algorithm;
> + uint16_t rss_key_size;
> + uint16_t rss_lut_size;
> +
> + uint16_t sw_idx; /* SW idx */
> +
> + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
> + uint16_t max_pkt_len; /* Maximum packet length */
> +
> + /* RSS info */
> + uint32_t *rss_lut;
> + uint8_t *rss_key;
> + uint64_t rss_hf;
> +
> + /* MSIX info*/
> + struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
> + uint16_t max_vectors;
> + struct virtchnl2_alloc_vectors *recv_vectors;
> +
> + /* Chunk info */
> + struct idpf_chunks_info chunks_info;
> +
> + /* Event from ipf */
> + bool link_up;
> + uint32_t link_speed;
> +
> + uint16_t devarg_id;
> + bool stopped;
> + struct virtchnl2_vport_stats eth_stats_offset; };
> +
> +struct idpf_adapter {
> + TAILQ_ENTRY(idpf_adapter) next;
> + struct iecm_hw hw;
> + char name[IDPF_ADAPTER_NAME_LEN];
> +
> + struct virtchnl_version_info virtchnl_version;
> + struct virtchnl2_get_capabilities *caps;
> +
> + volatile enum virtchnl_ops pend_cmd; /* pending command not
> finished */
> + uint32_t cmd_retval; /* return value of the cmd response from ipf */
> + uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf
> */
> +
> + uint32_t txq_model;
> + uint32_t rxq_model;
> +
> + /* Vport info */
> + uint8_t **vport_req_info;
> + uint8_t **vport_recv_info;
> + struct idpf_vport **vports;
> + uint16_t max_vport_nb;
> + uint16_t req_vports[IDPF_MAX_VPORT_NUM];
> + uint16_t req_vport_nb;
> + uint16_t cur_vports;
> + uint16_t cur_vport_nb;
> + uint16_t next_vport_idx;
> +
> + uint16_t used_vecs_num;
> +
> + /* Max config queue number per VC message */
> + uint32_t max_rxq_per_msg;
> + uint32_t max_txq_per_msg;
> +
> + uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
> +
> + bool stopped;
> +};
> +
> +TAILQ_HEAD(idpf_adapter_list, idpf_adapter); extern struct
> +idpf_adapter_list adapter_list;
> +
> +#define IDPF_DEV_TO_PCI(eth_dev) \
> + RTE_DEV_TO_PCI((eth_dev)->device)
> +
> +/* structure used for sending and checking response of virtchnl ops */
> +struct idpf_cmd_info {
> + uint32_t ops;
> + uint8_t *in_args; /* buffer for sending */
> + uint32_t in_args_size; /* buffer size for sending */
> + uint8_t *out_buffer; /* buffer for response */
> + uint32_t out_size; /* buffer size for response */
> +};
> +
> +/* notify current command done. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_notify_cmd(struct idpf_adapter *adapter, int msg_ret) {
> + adapter->cmd_retval = msg_ret;
> + rte_wmb();
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN; }
> +
> +/* clear current command. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_clear_cmd(struct idpf_adapter *adapter) {
> + rte_wmb();
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
> + adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS; }
> +
> +/* Check there is pending cmd in execution. If none, set new command.
> +*/ static inline int _atomic_set_cmd(struct idpf_adapter *adapter, enum
> +virtchnl_ops ops) {
> + int ret = rte_atomic32_cmpset(&adapter->pend_cmd,
> VIRTCHNL_OP_UNKNOWN,
> +ops);
> +
> + if (!ret)
> + PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter-
> >pend_cmd);
> +
> + return !ret;
> +}
> +
> +struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
> +void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev); int
> +idpf_check_api_version(struct idpf_adapter *adapter); int
> +idpf_get_caps(struct idpf_adapter *adapter); int
> +idpf_create_vport(struct rte_eth_dev *dev); int
> +idpf_destroy_vport(struct idpf_vport *vport); int
> +idpf_ena_dis_vport(struct idpf_vport *vport, bool enable); int
> +idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
> + uint16_t buf_len, uint8_t *buf);
> +
> +#endif /* _IDPF_ETHDEV_H_ */
> diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c new
> file mode 100644 index 0000000000..4708fb0666
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_vchnl.c
> @@ -0,0 +1,495 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <stdio.h>
> +#include <errno.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <stdarg.h>
> +#include <inttypes.h>
> +#include <rte_byteorder.h>
> +#include <rte_common.h>
> +
> +#include <rte_debug.h>
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#include "base/iecm_prototype.h"
> +
> +#define IDPF_CTLQ_LEN 64
> +
> +static int
> +idpf_vc_clean(struct idpf_adapter *adapter) {
> + struct iecm_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
> + uint16_t num_q_msg = IDPF_CTLQ_LEN;
> + struct iecm_dma_mem *dma_mem;
> + int err = 0;
> + uint32_t i;
> +
> + for (i = 0; i < 10; i++) {
> + err = iecm_ctlq_clean_sq(adapter->hw.asq, &num_q_msg,
> q_msg);
> + msleep(20);
> + if (num_q_msg)
> + break;
> + }
> + if (err)
> + goto error;
> +
> + /* Empty queue is not an error */
> + for (i = 0; i < num_q_msg; i++) {
> + dma_mem = q_msg[i]->ctx.indirect.payload;
> + if (dma_mem) {
> + iecm_free_dma_mem(&adapter->hw, dma_mem);
> + rte_free(dma_mem);
> + }
> + rte_free(q_msg[i]);
> + }
> +
> +error:
> + return err;
> +}
> +
> +static int
> +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
> + uint16_t msg_size, uint8_t *msg)
> +{
> + struct iecm_ctlq_msg *ctlq_msg;
> + struct iecm_dma_mem *dma_mem;
> + int err = 0;
> +
> + err = idpf_vc_clean(adapter);
> + if (err)
> + goto err;
> +
> + ctlq_msg = (struct iecm_ctlq_msg *)rte_zmalloc(NULL,
> + sizeof(struct iecm_ctlq_msg), 0);
> + if (!ctlq_msg) {
> + err = -ENOMEM;
> + goto err;
> + }
> +
> + dma_mem = (struct iecm_dma_mem *)rte_zmalloc(NULL,
> + sizeof(struct iecm_dma_mem), 0);
> + if (!dma_mem) {
> + err = -ENOMEM;
> + goto dma_mem_error;
> + }
> +
> + dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
> + iecm_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
> + if (!dma_mem->va) {
> + err = -ENOMEM;
> + goto dma_alloc_error;
> + }
> +
> + memcpy(dma_mem->va, msg, msg_size);
> +
> + ctlq_msg->opcode = iecm_mbq_opc_send_msg_to_pf;
> + ctlq_msg->func_id = 0;
> + ctlq_msg->data_len = msg_size;
> + ctlq_msg->cookie.mbx.chnl_opcode = op;
> + ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
> + ctlq_msg->ctx.indirect.payload = dma_mem;
> +
> + err = iecm_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
> + if (err)
> + goto send_error;
> +
> + return err;
> +
> +send_error:
> + iecm_free_dma_mem(&adapter->hw, dma_mem);
> +dma_alloc_error:
> + rte_free(dma_mem);
> +dma_mem_error:
> + rte_free(ctlq_msg);
> +err:
> + return err;
> +}
> +
> +static enum idpf_vc_result
> +idpf_read_msg_from_ipf(struct idpf_adapter *adapter, uint16_t buf_len,
> + uint8_t *buf)
> +{
> + struct iecm_hw *hw = &adapter->hw;
> + struct iecm_ctlq_msg ctlq_msg;
> + struct iecm_dma_mem *dma_mem = NULL;
> + enum idpf_vc_result result = IDPF_MSG_NON;
> + enum virtchnl_ops opcode;
> + uint16_t pending = 1;
> + int ret;
> +
> + ret = iecm_ctlq_recv(hw->arq, &pending, &ctlq_msg);
> + if (ret) {
> + PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
> + if (ret != IECM_ERR_CTLQ_NO_WORK)
> + result = IDPF_MSG_ERR;
> + return result;
> + }
> +
> + rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
> +
> + opcode = (enum
> virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
> + adapter->cmd_retval =
> + (enum
> +virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
> +
> + PMD_DRV_LOG(DEBUG, "CQ from ipf carries opcode %u, retval %d",
> + opcode, adapter->cmd_retval);
> +
> + if (opcode == VIRTCHNL2_OP_EVENT) {
> + struct virtchnl2_event *ve =
> + (struct virtchnl2_event
> *)ctlq_msg.ctx.indirect.payload->va;
> +
> + result = IDPF_MSG_SYS;
> + switch (ve->event) {
> + case VIRTCHNL2_EVENT_LINK_CHANGE:
> + /* TBD */
> + break;
> + default:
> + PMD_DRV_LOG(ERR, "%s: Unknown event %d from
> ipf",
> + __func__, ve->event);
> + break;
> + }
> + } else {
> + /* async reply msg on command issued by pf previously */
> + result = IDPF_MSG_CMD;
> + if (opcode != adapter->pend_cmd) {
> + PMD_DRV_LOG(WARNING, "command mismatch,
> expect %u, get %u",
> + adapter->pend_cmd, opcode);
> + result = IDPF_MSG_ERR;
> + }
> + }
> +
> + if (ctlq_msg.data_len)
> + dma_mem = ctlq_msg.ctx.indirect.payload;
> + else
> + pending = 0;
> +
> + ret = iecm_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
> + if (ret && dma_mem)
> + iecm_free_dma_mem(hw, dma_mem);
> +
> + return result;
> +}
> +
> +#define MAX_TRY_TIMES 200
> +#define ASQ_DELAY_MS 10
> +
> +int
> +idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t
> buf_len,
> + uint8_t *buf)
> +{
> + int ret, err = 0, i = 0;
> +
> + do {
> + ret = idpf_read_msg_from_ipf(adapter, buf_len, buf);
> + if (ret == IDPF_MSG_CMD)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + } while (i++ < MAX_TRY_TIMES);
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure (%d) for
> cmd %d",
> + adapter->cmd_retval, ops);
> + }
> +
> + return err;
> +}
> +
> +static int
> +idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info
> +*args) {
> + int err = 0;
> + int i = 0;
> + int ret;
> +
> + if (_atomic_set_cmd(adapter, args->ops))
> + return -1;
> +
> + ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args-
> >in_args);
> + if (ret) {
> + PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
> + _clear_cmd(adapter);
> + return ret;
> + }
> +
> + switch (args->ops) {
> + case VIRTCHNL_OP_VERSION:
> + case VIRTCHNL2_OP_GET_CAPS:
> + case VIRTCHNL2_OP_CREATE_VPORT:
> + case VIRTCHNL2_OP_DESTROY_VPORT:
> + case VIRTCHNL2_OP_SET_RSS_KEY:
> + case VIRTCHNL2_OP_SET_RSS_LUT:
> + case VIRTCHNL2_OP_SET_RSS_HASH:
> + case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
> + case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_QUEUES:
> + case VIRTCHNL2_OP_DISABLE_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_VPORT:
> + case VIRTCHNL2_OP_DISABLE_VPORT:
> + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
> + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
> + case VIRTCHNL2_OP_ALLOC_VECTORS:
> + case VIRTCHNL2_OP_DEALLOC_VECTORS:
> + case VIRTCHNL2_OP_GET_STATS:
> + /* for init virtchnl ops, need to poll the response */
> + err = idpf_read_one_msg(adapter, args->ops, args->out_size,
> args->out_buffer);
> + _clear_cmd(adapter);
> + break;
> + case VIRTCHNL2_OP_GET_PTYPE_INFO:
> + /* for multuple response message,
> + * do not handle the response here.
> + */
> + break;
> + default:
> + /* For other virtchnl ops in running time,
> + * wait for the cmd done flag.
> + */
> + do {
> + if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + /* If don't read msg or read sys event, continue */
> + } while (i++ < MAX_TRY_TIMES);
> + /* If there's no response is received, clear command */
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure
> (%d) for cmd %d",
> + adapter->cmd_retval, args->ops);
> + _clear_cmd(adapter);
> + }
> + break;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_check_api_version(struct idpf_adapter *adapter) {
> + struct virtchnl_version_info version;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&version, 0, sizeof(struct virtchnl_version_info));
> + version.major = VIRTCHNL_VERSION_MAJOR_2;
> + version.minor = VIRTCHNL_VERSION_MINOR_0;
> +
> + args.ops = VIRTCHNL_OP_VERSION;
> + args.in_args = (uint8_t *)&version;
> + args.in_args_size = sizeof(version);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL_OP_VERSION");
> + return err;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_get_caps(struct idpf_adapter *adapter) {
> + struct virtchnl2_get_capabilities caps_msg;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> + caps_msg.csum_caps =
> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> +
> + caps_msg.seg_caps =
> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> + VIRTCHNL2_CAP_SEG_GENERIC;
> +
> + caps_msg.rss_caps =
> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> +
> + caps_msg.hsplit_caps =
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
> +
> + caps_msg.rsc_caps =
> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
> +
> + caps_msg.other_caps =
> + VIRTCHNL2_CAP_RDMA |
> + VIRTCHNL2_CAP_SRIOV |
> + VIRTCHNL2_CAP_MACFILTER |
> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> + VIRTCHNL2_CAP_CRC |
> + VIRTCHNL2_CAP_WB_ON_ITR |
> + VIRTCHNL2_CAP_PROMISC |
> + VIRTCHNL2_CAP_LINK_SPEED |
> + VIRTCHNL2_CAP_VLAN;
> +
> + args.ops = VIRTCHNL2_OP_GET_CAPS;
> + args.in_args = (uint8_t *)&caps_msg;
> + args.in_args_size = sizeof(caps_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL2_OP_GET_CAPS");
> + return err;
> + }
> +
> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> +
> + return err;
> +}
> +
> +int
> +idpf_create_vport(struct rte_eth_dev *dev) {
> + struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
> + uint16_t idx = adapter->next_vport_idx;
> + struct virtchnl2_create_vport *vport_req_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_req_info[idx];
> + struct virtchnl2_create_vport vport_msg;
> + struct idpf_cmd_info args;
> + int err = -1;
> +
> + memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
> + vport_msg.vport_type = vport_req_info->vport_type;
> + vport_msg.txq_model = vport_req_info->txq_model;
> + vport_msg.rxq_model = vport_req_info->rxq_model;
> + vport_msg.num_tx_q = vport_req_info->num_tx_q;
> + vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
> + vport_msg.num_rx_q = vport_req_info->num_rx_q;
> + vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_CREATE_VPORT;
> + args.in_args = (uint8_t *)&vport_msg;
> + args.in_args_size = sizeof(vport_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL2_OP_CREATE_VPORT");
> + return err;
> + }
> +
> + if (!adapter->vport_recv_info[idx]) {
> + adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
> + IDPF_DFLT_MBX_BUF_SIZE,
> 0);
> + if (!adapter->vport_recv_info[idx]) {
> + PMD_INIT_LOG(ERR, "Failed to alloc
> vport_recv_info.");
> + return err;
> + }
> + }
> + rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
> + IDPF_DFLT_MBX_BUF_SIZE);
> + return err;
> +}
> +
> +int
> +idpf_destroy_vport(struct idpf_vport *vport) {
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
> + args.in_args = (uint8_t *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of
> VIRTCHNL2_OP_DESTROY_VPORT");
> + return err;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_ena_dis_vport(struct idpf_vport *vport, bool enable) {
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
> + VIRTCHNL2_OP_DISABLE_VPORT;
> + args.in_args = (u8 *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of
> VIRTCHNL2_OP_%s_VPORT",
> + enable ? "ENABLE" : "DISABLE");
> + }
> +
> + return err;
> +}
> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> new file mode 100644 index 0000000000..7d776d3a15
> --- /dev/null
> +++ b/drivers/net/idpf/meson.build
> @@ -0,0 +1,18 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022 Intel
> +Corporation
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
We should use 4 spaces instead of a tab in meson files.
> +endif
> +
> +subdir('base')
> +objs = [base_objs]
> +
> +sources = files(
> + 'idpf_ethdev.c',
> + 'idpf_vchnl.c',
Ditto.
> +)
> +
> +includes += include_directories('base')
> \ No newline at end of file
> diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
> new file mode 100644 index 0000000000..b7da224860
> --- /dev/null
> +++ b/drivers/net/idpf/version.map
> @@ -0,0 +1,3 @@
> +DPDK_22 {
> + local: *;
> +};
> \ No newline at end of file
> diff --git a/drivers/net/meson.build b/drivers/net/meson.build index
> e35652fe63..8faf8120c2 100644
> --- a/drivers/net/meson.build
> +++ b/drivers/net/meson.build
> @@ -28,6 +28,7 @@ drivers = [
> 'i40e',
> 'iavf',
> 'ice',
> + 'idpf',
Ditto.
> 'igc',
> 'ionic',
> 'ipn3ke',
> --
> 2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 14/14] net/idpf: add support for timestamp offload
2022-09-05 10:58 ` [PATCH v2 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-03 14:22 ` Andrew Rybchenko
@ 2022-10-10 7:56 ` Wu, Wenjun1
1 sibling, 0 replies; 376+ messages in thread
From: Wu, Wenjun1 @ 2022-10-10 7:56 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Guo, Junfeng, Qiao, Wenjing
> -----Original Message-----
> From: Junfeng Guo <junfeng.guo@intel.com>
> Sent: Monday, September 5, 2022 6:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Guo, Junfeng
> <junfeng.guo@intel.com>; Qiao, Wenjing <wenjing.qiao@intel.com>
> Subject: [PATCH v2 14/14] net/idpf: add support for timestamp offload
>
> Add support for timestamp offload.
>
> Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.h | 3 ++
> drivers/net/idpf/idpf_rxtx.c | 79 ++++++++++++++++++++++++++++++
> drivers/net/idpf/idpf_rxtx.h | 89 +++++++++++++++++++++++++++++++++-
> 3 files changed, 170 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> index a32d5758ac..968e0e3cbf 100644
> --- a/drivers/net/idpf/idpf_ethdev.h
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -184,6 +184,9 @@ struct idpf_adapter {
> bool tx_vec_allowed;
> bool rx_use_avx512;
> bool tx_use_avx512;
> +
> + /* For PTP */
> + uint64_t time_hw;
> };
>
> TAILQ_HEAD(idpf_adapter_list, idpf_adapter); diff --git
> a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index
> e31d202646..b0037eca08 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -10,6 +10,8 @@
> #include "idpf_rxtx.h"
> #include "idpf_rxtx_vec_common.h"
>
> +static int idpf_timestamp_dynfield_offset = -1;
> +
> const uint32_t *
> idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
> { @@ -965,6 +967,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev,
> uint16_t queue_idx,
> socket_id, tx_conf);
> }
>
> +static int
> +idpf_register_ts_mbuf(struct idpf_rx_queue *rxq) {
> + int err;
> + if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
> + /* Register mbuf field and flag for Rx timestamp */
> + err = rte_mbuf_dyn_rx_timestamp_register(
> + &idpf_timestamp_dynfield_offset,
> + &idpf_timestamp_dynflag);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Cannot register mbuf field/flag for
> timestamp");
> + return -EINVAL;
> + }
> + }
> + return 0;
> +}
> +
> static int
> idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq) { @@ -992,6
> +1012,10 @@ idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
> rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq-
> >rx_ring))[i];
> rxd->pkt_addr = dma_addr;
> rxd->hdr_addr = 0;
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + rxd->rsvd1 = 0;
> + rxd->rsvd2 = 0;
> +#endif
It seems that IDPF does not add any configuration related to descriptor length.
The descriptor length is always the default 32B, so the conditional compilation
is not necessary.
>
> rxq->sw_ring[i] = mbuf;
> }
> @@ -1057,6 +1081,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev,
> uint16_t rx_queue_id)
> return -EINVAL;
> }
>
> + err = idpf_register_ts_mbuf(rxq);
> + if (err) {
> + PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
> + rx_queue_id);
> + return -EIO;
> + }
> +
> if (!rxq->bufq1) {
> /* Single queue */
> err = idpf_alloc_single_rxq_mbufs(rxq); @@ -1441,6
> +1472,12 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf
> **rx_pkts,
> nb_rx = 0;
> rxq = (struct idpf_rx_queue *)rx_queue;
>
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + uint64_t ts_ns;
> + struct iecm_hw *hw = &rxq->adapter->hw;
> + struct idpf_adapter *ad = rxq->adapter; #endif
> +
> if (unlikely(!rxq) || unlikely(!rxq->q_started))
> return nb_rx;
>
Ditto.
> @@ -1451,6 +1488,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
> ptype_tbl = rxq->adapter->ptype_tbl;
>
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
> + rxq->hw_register_set = 1;
> +#endif
> +
Ditto.
> while (nb_rx < nb_pkts) {
> rx_desc = &rx_desc_ring[rx_id];
>
> @@ -1507,6 +1549,19 @@ idpf_splitq_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> status_err0_qw1 = rx_desc->status_err0_qw1;
> pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
> pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + if (idpf_timestamp_dynflag > 0) {
> + /* timestamp */
> + ts_ns = idpf_tstamp_convert_32b_64b(hw, ad,
> + rxq-
> >hw_register_set,
> +
> rte_le_to_cpu_32(rx_desc->ts_high));
> + rxq->hw_register_set = 0;
> + *RTE_MBUF_DYNFIELD(rxm,
> + idpf_timestamp_dynfield_offset,
> + rte_mbuf_timestamp_t *) = ts_ns;
> + rxm->ol_flags |= idpf_timestamp_dynflag;
> + }
> +#endif
Ditto.
> rxm->ol_flags |= pkt_flags;
>
> rx_pkts[nb_rx++] = rxm;
> @@ -1778,6 +1833,10 @@ idpf_singleq_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> nb_hold = 0;
> rxq = rx_queue;
>
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + uint64_t ts_ns;
> +#endif
Ditto.
> +
> if (unlikely(!rxq) || unlikely(!rxq->q_started))
> return nb_rx;
>
> @@ -1785,6 +1844,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> rx_ring = rxq->rx_ring;
> ptype_tbl = rxq->adapter->ptype_tbl;
>
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + struct iecm_hw *hw = &rxq->adapter->hw;
> + struct idpf_adapter *ad = rxq->adapter;
> + if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
> + rxq->hw_register_set = 1;
> +#endif
Ditto.
> +
> while (nb_rx < nb_pkts) {
> rxdp = &rx_ring[rx_id];
> rx_status0 = rte_le_to_cpu_16(rxdp-
> >flex_nic_wb.status_error0);
> @@ -1841,6 +1907,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct
> rte_mbuf **rx_pkts,
> rxm->packet_type =
>
> ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_fla
> gs0) &
> VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + if (idpf_timestamp_dynflag > 0) {
> + /* timestamp */
> + ts_ns = idpf_tstamp_convert_32b_64b(hw, ad,
> + rxq-
> >hw_register_set,
> +
> rte_le_to_cpu_32(rxdp->flex_nic_wb.flex_ts.ts_high));
> + rxq->hw_register_set = 0;
> + *RTE_MBUF_DYNFIELD(rxm,
> + idpf_timestamp_dynfield_offset,
> + rte_mbuf_timestamp_t *) = ts_ns;
> + rxm->ol_flags |= idpf_timestamp_dynflag;
> + }
> +#endif
Ditto.
> rx_pkts[nb_rx++] = rxm;
> }
> rxq->rx_tail = rx_id;
> diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index
> decd0a98c2..6fcb441143 100644
> --- a/drivers/net/idpf/idpf_rxtx.h
> +++ b/drivers/net/idpf/idpf_rxtx.h
> @@ -15,6 +15,41 @@
> #include "base/virtchnl2_lan_desc.h"
> #include "idpf_ethdev.h"
>
> +/* MTS */
> +#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
> +#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
> +#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
> +#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
> +#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
> +#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
> +#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
> +#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
> +#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
> +#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
> +#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
> +#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
> +#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
> +#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
> +#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
> +
> +#define PF_TIMESYNC_BAR4_BASE 0x0E400000
> +#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
> +#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
> +#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
> +#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
> +
> +#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
> +#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
> +#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
> +#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
> +#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
> +#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
> +#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
> +
> +#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
> +#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
> +#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
> +
> /* In QLEN must be whole number of 32 descriptors. */
> #define IDPF_ALIGN_RING_DESC 32
> #define IDPF_MIN_RING_DESC 32
> @@ -66,6 +101,8 @@
> (sizeof(struct virtchnl2_ptype) + \
> (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)-
> >proto_id[0])))
>
> +extern uint64_t idpf_timestamp_dynflag;
> +
> struct idpf_rx_queue {
> struct idpf_adapter *adapter; /* the adapter this queue belongs to
> */
> struct rte_mempool *mp; /* mbuf pool to populate Rx
> ring */
> @@ -231,5 +268,55 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
>
> const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
>
> -#endif /* _IDPF_RXTX_H_ */
> +#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
> +/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
> +static inline uint64_t idpf_tstamp_convert_32b_64b(struct iecm_hw *hw,
> +struct idpf_adapter *ad,
> + uint32_t flag, uint32_t in_timestamp) {
> +/* TODO: timestamp for ACC */
> +#ifdef RTE_ARCH_ARM64
> + return 0;
> +#endif /* RTE_ARCH_ARM64 */
> +
> +#ifdef RTE_ARCH_X86_64
> + const uint64_t mask = 0xFFFFFFFF;
> + uint32_t hi, lo, lo2, delta;
> + uint64_t ns;
> +
> + if (flag) {
> + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0,
> PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0,
> PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
> + PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> + lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
> + /*
> + * On typical system, the delta between lo and lo2 is ~1000ns,
> + * so 10000 seems a large-enough but not overly-big guard
> band.
> + */
> + if (lo > (UINT32_MAX -
> IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
> + lo2 = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + else
> + lo2 = lo;
> +
> + if (lo2 < lo) {
> + lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> + hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
> + }
> +
> + ad->time_hw = ((uint64_t)hi << 32) | lo;
> + }
> +
> + delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
> + if (delta > (mask / 2)) {
> + delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
> + ns = ad->time_hw - delta;
> + } else {
> + ns = ad->time_hw + delta;
> + }
> +
> + return ns;
> +#endif /* RTE_ARCH_X86_64 */
> +}
>
> +#endif /* _IDPF_RXTX_H_ */
> --
> 2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model
2022-09-05 10:58 ` [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-03 14:20 ` Andrew Rybchenko
@ 2022-10-10 8:06 ` Wu, Wenjun1
1 sibling, 0 replies; 376+ messages in thread
From: Wu, Wenjun1 @ 2022-10-10 8:06 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei; +Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Guo, Junfeng <junfeng.guo@intel.com>
> Sent: Monday, September 5, 2022 6:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Guo, Junfeng
> <junfeng.guo@intel.com>; Wu, Wenjun1 <wenjun1.wu@intel.com>
> Subject: [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue
> model
>
> Add support of AVX512 vector data path for single queue model.
>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.h | 5 +
> drivers/net/idpf/idpf_rxtx.c | 137 ++++
> drivers/net/idpf/idpf_rxtx.h | 11 +
> drivers/net/idpf/idpf_rxtx_vec_avx512.c | 917 ++++++++++++++++++++++++
> drivers/net/idpf/idpf_rxtx_vec_common.h | 89 +++
> drivers/net/idpf/meson.build | 31 +-
> 6 files changed, 1189 insertions(+), 1 deletion(-) create mode 100644
> drivers/net/idpf/idpf_rxtx_vec_avx512.c
> create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
>
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> index f96867f3d5..a32d5758ac 100644
> --- a/drivers/net/idpf/idpf_ethdev.h
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -179,6 +179,11 @@ struct idpf_adapter {
> uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
>
> bool stopped;
> +
> + bool rx_vec_allowed;
> + bool tx_vec_allowed;
> + bool rx_use_avx512;
> + bool tx_use_avx512;
> };
>
> TAILQ_HEAD(idpf_adapter_list, idpf_adapter); diff --git
> a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c index
> 54d83a7c61..e31d202646 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -4,9 +4,11 @@
>
> #include <ethdev_driver.h>
> #include <rte_net.h>
> +#include <rte_vect.h>
>
> #include "idpf_ethdev.h"
> #include "idpf_rxtx.h"
> +#include "idpf_rxtx_vec_common.h"
>
> const uint32_t *
> idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
> @@ -2146,15 +2148,110 @@ idpf_prep_pkts(__rte_unused void *tx_queue,
> struct rte_mbuf **tx_pkts,
> return i;
> }
>
> +static void __rte_cold
> +release_rxq_mbufs_vec(struct idpf_rx_queue *rxq) {
> + const uint16_t mask = rxq->nb_rx_desc - 1;
> + uint16_t i;
> +
> + if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
> + return;
> +
> + /* free all mbufs that are valid in the ring */
> + if (rxq->rxrearm_nb == 0) {
> + for (i = 0; i < rxq->nb_rx_desc; i++) {
> + if (rxq->sw_ring[i])
> + rte_pktmbuf_free_seg(rxq->sw_ring[i]);
> + }
> + } else {
> + for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask)
> {
> + if (rxq->sw_ring[i])
> + rte_pktmbuf_free_seg(rxq->sw_ring[i]);
> + }
> + }
> +
> + rxq->rxrearm_nb = rxq->nb_rx_desc;
> +
> + /* set all entries to NULL */
> + memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq-
> >nb_rx_desc); }
> +
> +static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
> + .release_mbufs = release_rxq_mbufs_vec, };
> +
> +static inline int
> +idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq) {
> + uintptr_t p;
> + struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
> +
> + mb_def.nb_segs = 1;
> + mb_def.data_off = RTE_PKTMBUF_HEADROOM;
> + mb_def.port = rxq->port_id;
> + rte_mbuf_refcnt_set(&mb_def, 1);
> +
> + /* prevent compiler reordering: rearm_data covers previous fields */
> + rte_compiler_barrier();
> + p = (uintptr_t)&mb_def.rearm_data;
> + rxq->mbuf_initializer = *(uint64_t *)p;
> + return 0;
> +}
> +
> +int __rte_cold
> +idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq) {
> + rxq->ops = &def_singleq_rx_ops_vec;
> + return idpf_singleq_rx_vec_setup_default(rxq);
> +}
> +
> void
> idpf_set_rx_function(struct rte_eth_dev *dev) {
> struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *ad = vport->adapter;
> + struct idpf_rx_queue *rxq;
> + int i;
> +
> +#ifdef RTE_ARCH_X86
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + if (idpf_rx_vec_dev_check_default(dev) ==
> IDPF_VECTOR_PATH &&
> + rte_vect_get_max_simd_bitwidth() >=
> RTE_VECT_SIMD_128) {
> + ad->rx_vec_allowed = true;
> +
> + if (rte_vect_get_max_simd_bitwidth() >=
> RTE_VECT_SIMD_512) #ifdef
> +CC_AVX512_SUPPORT
> + if
> (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
> +
> rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
> + ad->rx_use_avx512 = true;
> +#else
> + PMD_DRV_LOG(NOTICE,
> + "AVX512 is not supported in build env");
> #endif /*
> +CC_AVX512_SUPPORT */
> + } else {
> + ad->rx_vec_allowed = false;
> + }
> + }
> +#endif /* RTE_ARCH_X86 */
>
> if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
> dev->rx_pkt_burst = idpf_splitq_recv_pkts;
> return;
> } else {
> +#ifdef RTE_ARCH_X86
> + if (ad->rx_vec_allowed) {
> + for (i = 0; i < dev->data->nb_tx_queues; i++) {
> + rxq = dev->data->rx_queues[i];
> + (void)idpf_singleq_rx_vec_setup(rxq);
> + }
> +#ifdef CC_AVX512_SUPPORT
> + if (ad->rx_use_avx512) {
> + dev->rx_pkt_burst =
> idpf_singleq_recv_pkts_avx512;
> + return;
> + }
> +#endif /* CC_AVX512_SUPPORT */
> + }
> +#endif /* RTE_ARCH_X86 */
> dev->rx_pkt_burst = idpf_singleq_recv_pkts;
> return;
> }
> @@ -2164,12 +2261,52 @@ void
> idpf_set_tx_function(struct rte_eth_dev *dev) {
> struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *ad = vport->adapter;
> + struct idpf_tx_queue *txq;
> + int i;
> +
> +#ifdef RTE_ARCH_X86
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + if (idpf_rx_vec_dev_check_default(dev) ==
> IDPF_VECTOR_PATH &&
> + rte_vect_get_max_simd_bitwidth() >=
> RTE_VECT_SIMD_128) {
> + ad->tx_vec_allowed = true;
> + if (rte_vect_get_max_simd_bitwidth() >=
> RTE_VECT_SIMD_512) #ifdef
> +CC_AVX512_SUPPORT
> + if
> (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
> +
> rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
> + ad->tx_use_avx512 = true;
> +#else
> + PMD_DRV_LOG(NOTICE,
> + "AVX512 is not supported in build
> env"); #endif /*
> +CC_AVX512_SUPPORT */
> + } else {
> + ad->tx_vec_allowed = false;
> + }
> + }
> +#endif /* RTE_ARCH_X86 */
>
> if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
> dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
> dev->tx_pkt_prepare = idpf_prep_pkts;
> return;
> } else {
> +#ifdef RTE_ARCH_X86
> + if (ad->tx_vec_allowed) {
> +#ifdef CC_AVX512_SUPPORT
> + if (ad->tx_use_avx512) {
> + for (i = 0; i < dev->data->nb_tx_queues; i++) {
> + txq = dev->data->tx_queues[i];
> + if (!txq)
> + continue;
> +
> idpf_singleq_tx_vec_setup_avx512(txq);
> + }
> + dev->tx_pkt_burst =
> idpf_singleq_xmit_pkts_avx512;
> + dev->tx_pkt_prepare = idpf_prep_pkts;
> + return;
> + }
> +#endif /* CC_AVX512_SUPPORT */
> + }
> +#endif /* RTE_ARCH_X86 */
> dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
> dev->tx_pkt_prepare = idpf_prep_pkts;
> return;
> diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h index
> 3ccf9efe50..decd0a98c2 100644
> --- a/drivers/net/idpf/idpf_rxtx.h
> +++ b/drivers/net/idpf/idpf_rxtx.h
> @@ -122,6 +122,10 @@ struct idpf_tx_entry {
> uint16_t last_id;
> };
>
> +struct idpf_tx_vec_entry {
> + struct rte_mbuf *mbuf;
> +};
> +
> /* Structure associated with each TX queue. */ struct idpf_tx_queue {
> const struct rte_memzone *mz; /* memzone for Tx
> ring */
> @@ -207,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue,
> struct rte_mbuf **rx_pkts,
> uint16_t nb_pkts);
> uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
> uint16_t nb_pkts);
> +uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf
> **rx_pkts,
> + uint16_t nb_pkts);
> uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts);
> uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts);
> +uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> + uint16_t nb_pkts);
> +int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
> uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts);
> +int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
> +
> void idpf_stop_queues(struct rte_eth_dev *dev);
>
> void idpf_set_rx_function(struct rte_eth_dev *dev); diff --git
> a/drivers/net/idpf/idpf_rxtx_vec_avx512.c
> b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
> new file mode 100644
> index 0000000000..70eb5e1e7e
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
> @@ -0,0 +1,917 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include "idpf_rxtx_vec_common.h"
> +
> +#include <rte_vect.h>
> +
> +#ifndef __INTEL_COMPILER
> +#pragma GCC diagnostic ignored "-Wcast-qual"
> +#endif
> +
> +#define IDPF_DESCS_PER_LOOP_AVX 8
> +#define PKTLEN_SHIFT 10
> +
> +/**************************************************************
> ********
> +********
> + * If user knows a specific offload is not enabled by APP,
> + * the macro can be commented to save the effort of fast path.
> + * Currently below 2 features are supported in RX path,
> + * 1, checksum offload
> + * 2, VLAN/QINQ stripping
> + * 3, RSS hash
> + * 4, packet type analysis
> + * 5, flow director ID report
> +
> +***************************************************************
> ********
> +*******/
> +#define IDPF_RX_PTYPE_OFFLOAD
The comments should be reworked since it does not fully match the code.
> +
> +static __rte_always_inline void
> +idpf_singleq_rearm_common(struct idpf_rx_queue *rxq) {
> + struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
> + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
> + uint16_t rx_id;
> + int i;
> +
> + rxdp += rxq->rxrearm_start;
> +
> + /* Pull 'n' more MBUFs into the software ring */
> + if (rte_mempool_get_bulk(rxq->mp,
> + (void *)rxp,
> + IDPF_RXQ_REARM_THRESH) < 0) {
> + if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
> + rxq->nb_rx_desc) {
> + __m128i dma_addr0;
> +
> + dma_addr0 = _mm_setzero_si128();
> + for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
> + rxp[i] = &rxq->fake_mbuf;
> + _mm_store_si128((__m128i *)&rxdp[i].read,
> + dma_addr0);
> + }
> + }
> + rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
> + IDPF_RXQ_REARM_THRESH;
> + return;
> + }
> + struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
> + struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
> + __m512i dma_addr0_3, dma_addr4_7;
> + __m512i hdr_room =
> _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
> + /* Initialize the mbufs in vector, process 8 mbufs in one loop */
> + for (i = 0; i < IDPF_RXQ_REARM_THRESH;
> + i += 8, rxp += 8, rxdp += 8) {
> + __m128i vaddr0, vaddr1, vaddr2, vaddr3;
> + __m128i vaddr4, vaddr5, vaddr6, vaddr7;
> + __m256i vaddr0_1, vaddr2_3;
> + __m256i vaddr4_5, vaddr6_7;
> + __m512i vaddr0_3, vaddr4_7;
> +
> + mb0 = rxp[0];
> + mb1 = rxp[1];
> + mb2 = rxp[2];
> + mb3 = rxp[3];
> + mb4 = rxp[4];
> + mb5 = rxp[5];
> + mb6 = rxp[6];
> + mb7 = rxp[7];
> +
> + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
> + offsetof(struct rte_mbuf, buf_addr) + 8);
> + vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
> + vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
> + vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
> + vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
> + vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
> + vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
> + vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
> + vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
> +
> + /**
> + * merge 0 & 1, by casting 0 to 256-bit and inserting 1
> + * into the high lanes. Similarly for 2 & 3, and so on.
> + */
> + vaddr0_1 =
> +
> _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
> + vaddr1, 1);
> + vaddr2_3 =
> +
> _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
> + vaddr3, 1);
> + vaddr4_5 =
> +
> _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
> + vaddr5, 1);
> + vaddr6_7 =
> +
> _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
> + vaddr7, 1);
> + vaddr0_3 =
> +
> _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
> + vaddr2_3, 1);
> + vaddr4_7 =
> +
> _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
> + vaddr6_7, 1);
> +
> + /* convert pa to dma_addr hdr/data */
> + dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3,
> vaddr0_3);
> + dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7,
> vaddr4_7);
> +
> + /* add headroom to pa values */
> + dma_addr0_3 = _mm512_add_epi64(dma_addr0_3,
> hdr_room);
> + dma_addr4_7 = _mm512_add_epi64(dma_addr4_7,
> hdr_room);
> +
> + /* flush desc with pa dma_addr */
> + _mm512_store_si512((__m512i *)&rxdp->read,
> dma_addr0_3);
> + _mm512_store_si512((__m512i *)&(rxdp + 4)->read,
> dma_addr4_7);
> + }
> +
> + rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
> + if (rxq->rxrearm_start >= rxq->nb_rx_desc)
> + rxq->rxrearm_start = 0;
> +
> + rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
> +
> + rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
> + (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
> +
> + /* Update the tail pointer on the NIC */
> + IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id); }
> +
> +static __rte_always_inline void
> +idpf_singleq_rearm(struct idpf_rx_queue *rxq) {
> + int i;
> + uint16_t rx_id;
> + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
> + struct rte_mempool_cache *cache =
> + rte_mempool_default_cache(rxq->mp, rte_lcore_id());
> + struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
> +
> + rxdp += rxq->rxrearm_start;
> +
> + if (unlikely(!cache))
> + return idpf_singleq_rearm_common(rxq);
> +
> + /* We need to pull 'n' more MBUFs into the software ring from
> mempool
> + * We inline the mempool function here, so we can vectorize the
> copy
> + * from the cache into the shadow ring.
> + */
> +
> + /* Can this be satisfied from the cache? */
> + if (cache->len < IDPF_RXQ_REARM_THRESH) {
> + /* No. Backfill the cache first, and then fill from it */
> + uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
> + cache->len);
> +
> + /* How many do we require i.e. number to fill the cache + the
> request */
> + int ret = rte_mempool_ops_dequeue_bulk
> + (rxq->mp, &cache->objs[cache->len], req);
> + if (ret == 0) {
> + cache->len += req;
> + } else {
> + if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
> + rxq->nb_rx_desc) {
> + __m128i dma_addr0;
> +
> + dma_addr0 = _mm_setzero_si128();
> + for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP;
> i++) {
> + rxp[i] = &rxq->fake_mbuf;
> + _mm_storeu_si128((__m128i
> *)&rxdp[i].read,
> + dma_addr0);
> + }
> + }
> + rte_eth_devices[rxq->port_id].data-
> >rx_mbuf_alloc_failed +=
> + IDPF_RXQ_REARM_THRESH;
> + return;
> + }
> + }
> +
> + const __m512i iova_offsets = _mm512_set1_epi64(offsetof
> + (struct rte_mbuf,
> buf_iova));
> + const __m512i headroom =
> _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
> +
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
> + /* to shuffle the addresses to correct slots. Values 4-7 will contain
> + * zeros, so use 7 for a zero-value.
> + */
> + const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
> +#else
> + const __m512i permute_idx = _mm512_set_epi64(7, 3, 6, 2, 5, 1, 4, 0);
> +#endif
Conditional compilation is not necessary here.
> +
> + /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
> + * from mempool cache and populating both shadow and HW rings
> + */
> + for (i = 0; i < IDPF_RXQ_REARM_THRESH /
> IDPF_DESCS_PER_LOOP_AVX; i++) {
> + const __m512i mbuf_ptrs = _mm512_loadu_si512
> + (&cache->objs[cache->len -
> IDPF_DESCS_PER_LOOP_AVX]);
> + _mm512_storeu_si512(rxp, mbuf_ptrs);
> +
> + const __m512i iova_base_addrs = _mm512_i64gather_epi64
> + (_mm512_add_epi64(mbuf_ptrs,
> iova_offsets),
> + 0, /* base */
> + 1 /* scale */);
> + const __m512i iova_addrs =
> _mm512_add_epi64(iova_base_addrs,
> + headroom);
> +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
Conditional compilation is not necessary here.
> + const __m512i iovas0 = _mm512_castsi256_si512
> + (_mm512_extracti64x4_epi64(iova_addrs, 0));
> + const __m512i iovas1 = _mm512_castsi256_si512
> + (_mm512_extracti64x4_epi64(iova_addrs, 1));
> +
> + /* permute leaves desc 2-3 addresses in header address slots
> 0-1
> + * but these are ignored by driver since header split not
> + * enabled. Similarly for desc 6 & 7.
> + */
> + const __m512i desc0_1 = _mm512_permutexvar_epi64
> + (permute_idx,
> + iovas0);
> + const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
> +
> + const __m512i desc4_5 = _mm512_permutexvar_epi64
> + (permute_idx,
> + iovas1);
> + const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
> +
> + _mm512_storeu_si512((void *)rxdp, desc0_1);
> + _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
> + _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
> + _mm512_storeu_si512((void *)(rxdp + 6), desc6_7); #else
> + /* permute leaves desc 4-7 addresses in header address slots
> 0-3
> + * but these are ignored by driver since header split not
> + * enabled.
> + */
> + const __m512i desc0_3 =
> _mm512_permutexvar_epi64(permute_idx,
> + iova_addrs);
> + const __m512i desc4_7 = _mm512_bsrli_epi128(desc0_3, 8);
> +
> + _mm512_storeu_si512((void *)rxdp, desc0_3);
> + _mm512_storeu_si512((void *)(rxdp + 4), desc4_7); #endif
> + rxp += IDPF_DESCS_PER_LOOP_AVX;
> + rxdp += IDPF_DESCS_PER_LOOP_AVX;
> + cache->len -= IDPF_DESCS_PER_LOOP_AVX;
> + }
> +
> + rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
> + if (rxq->rxrearm_start >= rxq->nb_rx_desc)
> + rxq->rxrearm_start = 0;
> +
> + rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
> +
> + rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
> + (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
> +
> + /* Update the tail pointer on the NIC */
> + IECM_PCI_REG_WRITE(rxq->qrx_tail, rx_id); }
> +
> +#define IDPF_RX_LEN_MASK 0x80808080
> +static __rte_always_inline uint16_t
> +_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
> + struct rte_mbuf **rx_pkts,
> + uint16_t nb_pkts)
> +{
> +#ifdef IDPF_RX_PTYPE_OFFLOAD
> + const uint32_t *type_table = rxq->adapter->ptype_tbl; #endif
> +
> + const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
> + rxq->mbuf_initializer);
> + struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
> + volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
> +
> + rxdp += rxq->rx_tail;
> +
> + rte_prefetch0(rxdp);
> +
> + /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
> + nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
> +
> + /* See if we need to rearm the RX queue - gives the prefetch a bit
> + * of time to act
> + */
> + if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
> + idpf_singleq_rearm(rxq);
> +
> + /* Before we start moving massive data around, check to see if
> + * there is actually a packet available
> + */
> + if (!(rxdp->flex_nic_wb.status_error0 &
> + rte_cpu_to_le_32(1 <<
> VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
> + return 0;
> +
> + /* constants used in processing loop */
> + const __m512i crc_adjust =
> + _mm512_set_epi32
> + (/* 1st descriptor */
> + 0, /* ignore non-length fields */
> + -rxq->crc_len, /* sub crc on data_len */
> + -rxq->crc_len, /* sub crc on pkt_len */
> + 0, /* ignore pkt_type field */
> + /* 2nd descriptor */
> + 0, /* ignore non-length fields */
> + -rxq->crc_len, /* sub crc on data_len */
> + -rxq->crc_len, /* sub crc on pkt_len */
> + 0, /* ignore pkt_type field */
> + /* 3rd descriptor */
> + 0, /* ignore non-length fields */
> + -rxq->crc_len, /* sub crc on data_len */
> + -rxq->crc_len, /* sub crc on pkt_len */
> + 0, /* ignore pkt_type field */
> + /* 4th descriptor */
> + 0, /* ignore non-length fields */
> + -rxq->crc_len, /* sub crc on data_len */
> + -rxq->crc_len, /* sub crc on pkt_len */
> + 0 /* ignore pkt_type field */
> + );
> +
> + /* 8 packets DD mask, LSB in each 32-bit value */
> + const __m256i dd_check = _mm256_set1_epi32(1);
> +
> + /* mask to shuffle from desc. to mbuf (4 descriptors)*/
> + const __m512i shuf_msk =
> + _mm512_set_epi32
> + (/* 1st descriptor */
> + 0xFFFFFFFF, /* octet 4~7, 32bits rss */
> + 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
> + /* octet 15~14, 16 bits data_len */
> + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out
> */
> + /* octet 15~14, low 16 bits pkt_len */
> + 0xFFFFFFFF, /* pkt_type set as unknown */
> + /* 2nd descriptor */
> + 0xFFFFFFFF, /* octet 4~7, 32bits rss */
> + 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
> + /* octet 15~14, 16 bits data_len */
> + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out
> */
> + /* octet 15~14, low 16 bits pkt_len */
> + 0xFFFFFFFF, /* pkt_type set as unknown */
> + /* 3rd descriptor */
> + 0xFFFFFFFF, /* octet 4~7, 32bits rss */
> + 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
> + /* octet 15~14, 16 bits data_len */
> + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out
> */
> + /* octet 15~14, low 16 bits pkt_len */
> + 0xFFFFFFFF, /* pkt_type set as unknown */
> + /* 4th descriptor */
> + 0xFFFFFFFF, /* octet 4~7, 32bits rss */
> + 0xFFFF0504, /* octet 2~3, low 16 bits vlan_macip */
> + /* octet 15~14, 16 bits data_len */
> + 0xFFFF0504, /* skip high 16 bits pkt_len, zero out
> */
> + /* octet 15~14, low 16 bits pkt_len */
> + 0xFFFFFFFF /* pkt_type set as unknown */
> + );
The comments should be reworked since it does not fully match the code.
> + /**
> + * compile-time check the above crc and shuffle layout is correct.
> + * NOTE: the first field (lowest address) is given last in set_epi
> + * calls above.
> + */
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
> + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
> + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
> + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
> + offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
> +
> + uint16_t i, received;
> +
> + for (i = 0, received = 0; i < nb_pkts;
> + i += IDPF_DESCS_PER_LOOP_AVX,
> + rxdp += IDPF_DESCS_PER_LOOP_AVX) {
> + /* step 1, copy over 8 mbuf pointers to rx_pkts array */
> + _mm256_storeu_si256((void *)&rx_pkts[i],
> + _mm256_loadu_si256((void *)&sw_ring[i]));
> #ifdef
> +RTE_ARCH_X86_64
> + _mm256_storeu_si256
> + ((void *)&rx_pkts[i + 4],
> + _mm256_loadu_si256((void *)&sw_ring[i + 4]));
> #endif
> +
> + __m512i raw_desc0_3, raw_desc4_7;
> + const __m128i raw_desc7 =
> + _mm_load_si128((void *)(rxdp + 7));
> + rte_compiler_barrier();
> + const __m128i raw_desc6 =
> + _mm_load_si128((void *)(rxdp + 6));
> + rte_compiler_barrier();
> + const __m128i raw_desc5 =
> + _mm_load_si128((void *)(rxdp + 5));
> + rte_compiler_barrier();
> + const __m128i raw_desc4 =
> + _mm_load_si128((void *)(rxdp + 4));
> + rte_compiler_barrier();
> + const __m128i raw_desc3 =
> + _mm_load_si128((void *)(rxdp + 3));
> + rte_compiler_barrier();
> + const __m128i raw_desc2 =
> + _mm_load_si128((void *)(rxdp + 2));
> + rte_compiler_barrier();
> + const __m128i raw_desc1 =
> + _mm_load_si128((void *)(rxdp + 1));
> + rte_compiler_barrier();
> + const __m128i raw_desc0 =
> + _mm_load_si128((void *)(rxdp + 0));
> +
> + raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
> + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7,
> raw_desc5, 1);
> + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7,
> raw_desc6, 2);
> + raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7,
> raw_desc7, 3);
> + raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
> + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3,
> raw_desc1, 1);
> + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3,
> raw_desc2, 2);
> + raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3,
> raw_desc3, 3);
> +
> + /**
> + * convert descriptors 4-7 into mbufs, adjusting length and
> + * re-arranging fields. Then write into the mbuf
> + */
> + const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
> + PKTLEN_SHIFT);
> + const __m512i desc4_7 =
> _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
> + raw_desc4_7,
> + len4_7);
> + __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
> +
> + mb4_7 = _mm512_add_epi32(mb4_7, crc_adjust); #ifdef
> +IDPF_RX_PTYPE_OFFLOAD
> + /**
> + * to get packet types, shift 64-bit values down 30 bits
> + * and so ptype is in lower 8-bits in each
> + */
> + const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
> + const __m256i ptypes6_7 =
> _mm512_extracti64x4_epi64(ptypes4_7, 1);
> + const __m256i ptypes4_5 =
> _mm512_extracti64x4_epi64(ptypes4_7, 0);
> + const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
> + const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
> + const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
> + const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
> +
> + const __m512i ptype4_7 = _mm512_set_epi32
> + (0, 0, 0, type_table[ptype7],
> + 0, 0, 0, type_table[ptype6],
> + 0, 0, 0, type_table[ptype5],
> + 0, 0, 0, type_table[ptype4]);
> + mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7,
> ptype4_7); #endif
> +
> + /**
> + * convert descriptors 0-3 into mbufs, adjusting length and
> + * re-arranging fields. Then write into the mbuf
> + */
> + const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
> + PKTLEN_SHIFT);
> + const __m512i desc0_3 =
> _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
> + raw_desc0_3,
> + len0_3);
> + __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
> +
> + mb0_3 = _mm512_add_epi32(mb0_3, crc_adjust); #ifdef
> +IDPF_RX_PTYPE_OFFLOAD
> + /* get the packet types */
> + const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
> + const __m256i ptypes2_3 =
> _mm512_extracti64x4_epi64(ptypes0_3, 1);
> + const __m256i ptypes0_1 =
> _mm512_extracti64x4_epi64(ptypes0_3, 0);
> + const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
> + const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
> + const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
> + const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
> +
> + const __m512i ptype0_3 = _mm512_set_epi32
> + (0, 0, 0, type_table[ptype3],
> + 0, 0, 0, type_table[ptype2],
> + 0, 0, 0, type_table[ptype1],
> + 0, 0, 0, type_table[ptype0]);
> + mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3,
> ptype0_3); #endif
> +
> + /**
> + * use permute/extract to get status content
> + * After the operations, the packets status flags are in the
> + * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
> + */
> + /* merge the status bits into one register */
> + const __m512i status_permute_msk = _mm512_set_epi32
> + (0, 0, 0, 0,
> + 0, 0, 0, 0,
> + 22, 30, 6, 14,
> + 18, 26, 2, 10);
> + const __m512i raw_status0_7 =
> _mm512_permutex2var_epi32
> + (raw_desc4_7, status_permute_msk, raw_desc0_3);
> + __m256i status0_7 = _mm512_extracti64x4_epi64
> + (raw_status0_7, 0);
> +
> + /* now do flag manipulation */
> +
> + /**
> + * At this point, we have the 8 sets of flags in the low 16-bits
> + * of each 32-bit value in vlan0.
> + * We want to extract these, and merge them with the mbuf
> init
> + * data so we can do a single write to the mbuf to set the
> flags
> + * and all the other initialization fields. Extracting the
> + * appropriate flags means that we have to do a shift and
> blend
> + * for each mbuf before we do the write. However, we can
> also
> + * add in the previously computed rx_descriptor fields to
> + * make a single 256-bit write per mbuf
> + */
> + /* check the structure matches expectations */
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
> + offsetof(struct rte_mbuf, rearm_data) + 8);
> + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf,
> rearm_data) !=
> + RTE_ALIGN(offsetof(struct rte_mbuf,
> + rearm_data),
> + 16));
> + /* build up data and do writes */
> + __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
> + rearm6, rearm7;
> + const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7,
> 0);
> + const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7,
> 1);
> + const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3,
> 0);
> + const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3,
> 1);
> +
> + rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7,
> 0x20);
> + rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5,
> 0x20);
> + rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3,
> 0x20);
> + rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1,
> 0x20);
> +
> + /* write to mbuf */
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]-
> >rearm_data,
> + rearm6);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]-
> >rearm_data,
> + rearm4);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]-
> >rearm_data,
> + rearm2);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]-
> >rearm_data,
> + rearm0);
> +
> + rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
> + rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
> + rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
> + rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
> +
> + /* again write to mbufs */
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]-
> >rearm_data,
> + rearm7);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]-
> >rearm_data,
> + rearm5);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]-
> >rearm_data,
> + rearm3);
> + _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]-
> >rearm_data,
> + rearm1);
> +
> + /* perform dd_check */
> + status0_7 = _mm256_and_si256(status0_7, dd_check);
> + status0_7 = _mm256_packs_epi32(status0_7,
> + _mm256_setzero_si256());
> +
> + uint64_t burst = __builtin_popcountll
> + (_mm_cvtsi128_si64
> + (_mm256_extracti128_si256
> + (status0_7, 1)));
> + burst += __builtin_popcountll
> + (_mm_cvtsi128_si64
> +
> (_mm256_castsi256_si128(status0_7)));
> + received += burst;
> + if (burst != IDPF_DESCS_PER_LOOP_AVX)
> + break;
> + }
> +
> + /* update tail pointers */
> + rxq->rx_tail += received;
> + rxq->rx_tail &= (rxq->nb_rx_desc - 1);
> + if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
> + rxq->rx_tail--;
> + received--;
> + }
> + rxq->rxrearm_nb += received;
> + return received;
> +}
> +
> +/**
> + * Notice:
> + * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet */ uint16_t
> +idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
> + uint16_t nb_pkts)
> +{
> + return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts,
> nb_pkts);
> +}
> +
> +static __rte_always_inline int
> +idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq) {
> + struct idpf_tx_vec_entry *txep;
> + uint32_t n;
> + uint32_t i;
> + int nb_free = 0;
> + struct rte_mbuf *m, *free[txq->rs_thresh];
> +
> + /* check DD bits on threshold descriptor */
> + if ((txq->tx_ring[txq->next_dd].qw1 &
> + rte_cpu_to_le_64(IECM_TXD_QW1_DTYPE_M)) !=
> +
> rte_cpu_to_le_64(IECM_TX_DESC_DTYPE_DESC_DONE))
> + return 0;
> +
> + n = txq->rs_thresh;
> +
> + /* first buffer to free from S/W ring is at index
> + * tx_next_dd - (tx_rs_thresh-1)
> + */
> + txep = (void *)txq->sw_ring;
> + txep += txq->next_dd - (n - 1);
> +
> + if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n
> & 31) == 0) {
> + struct rte_mempool *mp = txep[0].mbuf->pool;
> + struct rte_mempool_cache *cache =
> rte_mempool_default_cache(mp,
> +
> rte_lcore_id());
> + void **cache_objs;
> +
> + if (!cache || cache->len == 0)
> + goto normal;
> +
> + cache_objs = &cache->objs[cache->len];
> +
> + if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
> + rte_mempool_ops_enqueue_bulk(mp, (void *)txep,
> n);
> + goto done;
> + }
> +
> + /* The cache follows the following algorithm
> + * 1. Add the objects to the cache
> + * 2. Anything greater than the cache min value (if it crosses
> the
> + * cache flush threshold) is flushed to the ring.
> + */
> + /* Add elements back into the cache */
> + uint32_t copied = 0;
> + /* n is multiple of 32 */
> + while (copied < n) {
> + const __m512i a =
> _mm512_loadu_si512(&txep[copied]);
> + const __m512i b =
> _mm512_loadu_si512(&txep[copied + 8]);
> + const __m512i c =
> _mm512_loadu_si512(&txep[copied + 16]);
> + const __m512i d =
> _mm512_loadu_si512(&txep[copied + 24]);
> +
> + _mm512_storeu_si512(&cache_objs[copied], a);
> + _mm512_storeu_si512(&cache_objs[copied + 8], b);
> + _mm512_storeu_si512(&cache_objs[copied + 16], c);
> + _mm512_storeu_si512(&cache_objs[copied + 24], d);
> + copied += 32;
> + }
> + cache->len += n;
> +
> + if (cache->len >= cache->flushthresh) {
> + rte_mempool_ops_enqueue_bulk(mp,
> + &cache->objs[cache->size],
> + cache->len - cache->size);
> + cache->len = cache->size;
> + }
> + goto done;
> + }
> +
> +normal:
> + m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
> + if (likely(m)) {
> + free[0] = m;
> + nb_free = 1;
> + for (i = 1; i < n; i++) {
> + m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> + if (likely(m)) {
> + if (likely(m->pool == free[0]->pool)) {
> + free[nb_free++] = m;
> + } else {
> + rte_mempool_put_bulk(free[0]->pool,
> + (void *)free,
> + nb_free);
> + free[0] = m;
> + nb_free = 1;
> + }
> + }
> + }
> + rte_mempool_put_bulk(free[0]->pool, (void **)free,
> nb_free);
> + } else {
> + for (i = 1; i < n; i++) {
> + m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> + if (m)
> + rte_mempool_put(m->pool, m);
> + }
> + }
> +
> +done:
> + /* buffers were freed, update counters */
> + txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
> + txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
> + if (txq->next_dd >= txq->nb_tx_desc)
> + txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
> +
> + return txq->rs_thresh;
> +}
> +
> +static __rte_always_inline void
> +tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
> + struct rte_mbuf **tx_pkts, uint16_t nb_pkts) {
> + int i;
> +
> + for (i = 0; i < (int)nb_pkts; ++i)
> + txep[i].mbuf = tx_pkts[i];
> +}
> +
> +static __rte_always_inline void
> +idpf_vtx1(volatile struct iecm_base_tx_desc *txdp,
> + struct rte_mbuf *pkt, uint64_t flags) {
> + uint64_t high_qw =
> + (IECM_TX_DESC_DTYPE_DATA |
> + ((uint64_t)flags << IECM_TXD_QW1_CMD_S) |
> + ((uint64_t)pkt->data_len << IECM_TXD_QW1_TX_BUF_SZ_S));
> +
> + __m128i descriptor = _mm_set_epi64x(high_qw,
> + pkt->buf_iova + pkt->data_off);
> + _mm_storeu_si128((__m128i *)txdp, descriptor); }
> +
> +#define IDPF_TX_LEN_MASK 0xAA
> +#define IDPF_TX_OFF_MASK 0x55
> +static __rte_always_inline void
> +idpf_vtx(volatile struct iecm_base_tx_desc *txdp,
> + struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags) {
> + const uint64_t hi_qw_tmpl = (IECM_TX_DESC_DTYPE_DATA |
> + ((uint64_t)flags << IECM_TXD_QW1_CMD_S));
> +
> + /* if unaligned on 32-bit boundary, do one to align */
> + if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
> + idpf_vtx1(txdp, *pkt, flags);
> + nb_pkts--, txdp++, pkt++;
> + }
> +
> + /* do 4 at a time while possible, in bursts */
> + for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
> + uint64_t hi_qw3 =
> + hi_qw_tmpl |
> + ((uint64_t)pkt[3]->data_len <<
> + IECM_TXD_QW1_TX_BUF_SZ_S);
> + uint64_t hi_qw2 =
> + hi_qw_tmpl |
> + ((uint64_t)pkt[2]->data_len <<
> + IECM_TXD_QW1_TX_BUF_SZ_S);
> + uint64_t hi_qw1 =
> + hi_qw_tmpl |
> + ((uint64_t)pkt[1]->data_len <<
> + IECM_TXD_QW1_TX_BUF_SZ_S);
> + uint64_t hi_qw0 =
> + hi_qw_tmpl |
> + ((uint64_t)pkt[0]->data_len <<
> + IECM_TXD_QW1_TX_BUF_SZ_S);
> +
> + __m512i desc0_3 =
> + _mm512_set_epi64
> + (hi_qw3,
> + pkt[3]->buf_iova + pkt[3]->data_off,
> + hi_qw2,
> + pkt[2]->buf_iova + pkt[2]->data_off,
> + hi_qw1,
> + pkt[1]->buf_iova + pkt[1]->data_off,
> + hi_qw0,
> + pkt[0]->buf_iova + pkt[0]->data_off);
> + _mm512_storeu_si512((void *)txdp, desc0_3);
> + }
> +
> + /* do any last ones */
> + while (nb_pkts) {
> + idpf_vtx1(txdp, *pkt, flags);
> + txdp++, pkt++, nb_pkts--;
> + }
> +}
> +
> +static __rte_always_inline uint16_t
> +idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> + uint16_t nb_pkts)
> +{
> + struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
> + volatile struct iecm_base_tx_desc *txdp;
> + struct idpf_tx_vec_entry *txep;
> + uint16_t n, nb_commit, tx_id;
> + /* bit2 is reserved and must be set to 1 according to Spec */
> + uint64_t flags = IECM_TX_DESC_CMD_EOP | 0x04;
> + uint64_t rs = IECM_TX_DESC_CMD_RS | flags;
> +
> + /* cross rx_thresh boundary is not allowed */
> + nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
> +
> + if (txq->nb_free < txq->free_thresh)
> + idpf_tx_free_bufs_avx512(txq);
> +
> + nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
> + if (unlikely(nb_pkts == 0))
> + return 0;
> +
> + tx_id = txq->tx_tail;
> + txdp = &txq->tx_ring[tx_id];
> + txep = (void *)txq->sw_ring;
> + txep += tx_id;
> +
> + txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
> +
> + n = (uint16_t)(txq->nb_tx_desc - tx_id);
> + if (nb_commit >= n) {
> + tx_backlog_entry_avx512(txep, tx_pkts, n);
> +
> + idpf_vtx(txdp, tx_pkts, n - 1, flags);
> + tx_pkts += (n - 1);
> + txdp += (n - 1);
> +
> + idpf_vtx1(txdp, *tx_pkts++, rs);
> +
> + nb_commit = (uint16_t)(nb_commit - n);
> +
> + tx_id = 0;
> + txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
> +
> + /* avoid reach the end of ring */
> + txdp = &txq->tx_ring[tx_id];
> + txep = (void *)txq->sw_ring;
> + txep += tx_id;
> + }
> +
> + tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
> +
> + idpf_vtx(txdp, tx_pkts, nb_commit, flags);
> +
> + tx_id = (uint16_t)(tx_id + nb_commit);
> + if (tx_id > txq->next_rs) {
> + txq->tx_ring[txq->next_rs].qw1 |=
> +
> rte_cpu_to_le_64(((uint64_t)IECM_TX_DESC_CMD_RS) <<
> + IECM_TXD_QW1_CMD_S);
> + txq->next_rs =
> + (uint16_t)(txq->next_rs + txq->rs_thresh);
> + }
> +
> + txq->tx_tail = tx_id;
> +
> + IECM_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
> +
> + return nb_pkts;
> +}
> +
> +static __rte_always_inline uint16_t
> +idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf
> **tx_pkts,
> + uint16_t nb_pkts)
> +{
> + uint16_t nb_tx = 0;
> + struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
> +
> + while (nb_pkts) {
> + uint16_t ret, num;
> +
> + num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
> + ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue,
> &tx_pkts[nb_tx],
> + num);
> + nb_tx += ret;
> + nb_pkts -= ret;
> + if (ret < num)
> + break;
> + }
> +
> + return nb_tx;
> +}
> +
> +uint16_t
> +idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
> + uint16_t nb_pkts)
> +{
> + return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts,
> nb_pkts); }
> +
> +static inline void
> +idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq) {
> + unsigned int i;
> + const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
> + struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
> +
> + if (!txq->sw_ring || txq->nb_free == max_desc)
> + return;
> +
> + i = txq->next_dd - txq->rs_thresh + 1;
> + if (txq->tx_tail < i) {
> + for (; i < txq->nb_tx_desc; i++) {
> + rte_pktmbuf_free_seg(swr[i].mbuf);
> + swr[i].mbuf = NULL;
> + }
> + i = 0;
> + }
> +}
> +
> +static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
> + .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
> +};
> +
> +int __rte_cold
> +idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq) {
> + txq->ops = &avx512_singleq_tx_vec_ops;
> + return 0;
> +}
> diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h
> b/drivers/net/idpf/idpf_rxtx_vec_common.h
> new file mode 100644
> index 0000000000..336d4c8b25
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
> @@ -0,0 +1,89 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_RXTX_VEC_COMMON_H_
> +#define _IDPF_RXTX_VEC_COMMON_H_
> +#include <stdint.h>
> +#include <ethdev_driver.h>
> +#include <rte_malloc.h>
> +
> +#include "idpf_ethdev.h"
> +#include "idpf_rxtx.h"
> +
> +#ifndef __INTEL_COMPILER
> +#pragma GCC diagnostic ignored "-Wcast-qual"
> +#endif
> +
> +#define IDPF_VECTOR_PATH 0
> +
> +static inline int
> +idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq) {
> + if (!rxq)
> + return -1;
> +
> + if (!rte_is_power_of_2(rxq->nb_rx_desc))
> + return -1;
> +
> + if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
> + return -1;
> +
> + if (rxq->nb_rx_desc % rxq->rx_free_thresh)
> + return -1;
> +
> + /* Currently, vector path doesn't support timestamp. */
> + if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
> + return -1;
> +
> + return IDPF_VECTOR_PATH;
> +}
> +
> +static inline int
> +idpf_tx_vec_queue_default(struct idpf_tx_queue *txq) {
> + if (!txq)
> + return -1;
> +
> + if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
> + (txq->rs_thresh & 3) != 0)
> + return -1;
> +
> + return IDPF_VECTOR_PATH;
> +}
> +
> +static inline int
> +idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev) {
> + int i;
> + struct idpf_rx_queue *rxq;
> + int ret = 0;
> +
> + for (i = 0; i < dev->data->nb_rx_queues; i++) {
> + rxq = dev->data->rx_queues[i];
> + ret = (idpf_rx_vec_queue_default(rxq));
> + if (ret < 0)
> + return -1;
> + }
> +
> + return IDPF_VECTOR_PATH;
> +}
> +
> +static inline int
> +idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev) {
> + int i;
> + struct idpf_tx_queue *txq;
> + int ret = 0;
> +
> + for (i = 0; i < dev->data->nb_tx_queues; i++) {
> + txq = dev->data->tx_queues[i];
> + ret = idpf_tx_vec_queue_default(txq);
> + if (ret < 0)
> + return -1;
> + }
> +
> + return IDPF_VECTOR_PATH;
> +}
> +
> +#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> index 338a39e391..ee4bb94a97 100644
> --- a/drivers/net/idpf/meson.build
> +++ b/drivers/net/idpf/meson.build
> @@ -16,4 +16,33 @@ sources = files(
> 'idpf_vchnl.c',
> )
>
> -includes += include_directories('base') \ No newline at end of file
> +if arch_subdir == 'x86'
> + idpf_avx512_cpu_support = (
> + cc.get_define('__AVX512F__', args: machine_args) != '' and
> + cc.get_define('__AVX512BW__', args: machine_args) != ''
> + )
> +
> + idpf_avx512_cc_support = (
> + not machine_args.contains('-mno-avx512f') and
> + cc.has_argument('-mavx512f') and
> + cc.has_argument('-mavx512bw')
> + )
> +
> + if idpf_avx512_cpu_support == true or idpf_avx512_cc_support ==
> true
> + cflags += ['-DCC_AVX512_SUPPORT']
> + avx512_args = [cflags, '-mavx512f', '-mavx512bw']
> + if cc.has_argument('-march=skylake-avx512')
> + avx512_args += '-march=skylake-avx512'
> + endif
> + idpf_avx512_lib = static_library('idpf_avx512_lib',
> + 'idpf_rxtx_vec_avx512.c',
> + dependencies: [static_rte_ethdev, static_rte_bus_pci,
> + static_rte_kvargs, static_rte_hash],
> + include_directories: includes,
> + c_args: avx512_args)
> + objs +=
> idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
> + endif
> +endif
> +
> +includes += include_directories('base')
> +
4 spaces should be used instead of a tab.
> --
> 2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation
2022-09-05 10:58 ` [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation Junfeng Guo
2022-10-03 13:20 ` Andrew Rybchenko
@ 2022-10-12 8:07 ` Wu, Wenjun1
1 sibling, 0 replies; 376+ messages in thread
From: Wu, Wenjun1 @ 2022-10-12 8:07 UTC (permalink / raw)
To: Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Guo, Junfeng
> -----Original Message-----
> From: Junfeng Guo <junfeng.guo@intel.com>
> Sent: Monday, September 5, 2022 6:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Guo, Junfeng
> <junfeng.guo@intel.com>
> Subject: [PATCH v2 02/14] net/idpf/base: add logs and OS specific
> implementation
>
> Add PMD logs.
> Add some MACRO definations and small functions which are specific for
> DPDK.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/base/iecm_osdep.h | 365
> +++++++++++++++++++++++++++++
> drivers/net/idpf/idpf_logs.h | 38 +++
> 2 files changed, 403 insertions(+)
> create mode 100644 drivers/net/idpf/base/iecm_osdep.h
> create mode 100644 drivers/net/idpf/idpf_logs.h
>
> diff --git a/drivers/net/idpf/base/iecm_osdep.h
> b/drivers/net/idpf/base/iecm_osdep.h
> new file mode 100644
> index 0000000000..60e21fbc1b
> --- /dev/null
> +++ b/drivers/net/idpf/base/iecm_osdep.h
> @@ -0,0 +1,365 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2001-2022 Intel Corporation */
> +
> +#ifndef _IECM_OSDEP_H_
> +#define _IECM_OSDEP_H_
> +
> +#include <string.h>
> +#include <stdint.h>
> +#include <stdio.h>
> +#include <stdarg.h>
> +#include <inttypes.h>
> +#include <sys/queue.h>
> +#include <stdbool.h>
> +
> +#include <rte_common.h>
> +#include <rte_memcpy.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_byteorder.h>
> +#include <rte_cycles.h>
> +#include <rte_spinlock.h>
> +#include <rte_log.h>
> +#include <rte_random.h>
> +#include <rte_io.h>
> +
> +#include "../idpf_logs.h"
> +
> +#define INLINE inline
> +#define STATIC static
> +
> +typedef uint8_t u8;
> +typedef int8_t s8;
> +typedef uint16_t u16;
> +typedef int16_t s16;
> +typedef uint32_t u32;
> +typedef int32_t s32;
> +typedef uint64_t u64;
> +typedef uint64_t s64;
> +
> +typedef enum iecm_status iecm_status;
> +typedef struct iecm_lock iecm_lock;
> +
> +#define __iomem
> +#define hw_dbg(hw, S, A...) do {} while (0)
> +#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
> +#define lower_32_bits(n) ((u32)(n))
> +#define low_16_bits(x) ((x) & 0xFFFF)
> +#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
> +
> +#ifndef ETH_ADDR_LEN
> +#define ETH_ADDR_LEN 6
> +#endif
> +
> +#ifndef __le16
> +#define __le16 uint16_t
> +#endif
> +#ifndef __le32
> +#define __le32 uint32_t
> +#endif
> +#ifndef __le64
> +#define __le64 uint64_t
> +#endif
> +#ifndef __be16
> +#define __be16 uint16_t
> +#endif
> +#ifndef __be32
> +#define __be32 uint32_t
> +#endif
> +#ifndef __be64
> +#define __be64 uint64_t
> +#endif
> +
> +#ifndef __always_unused
> +#define __always_unused __attribute__((__unused__)) #endif #ifndef
> +__maybe_unused #define __maybe_unused __attribute__((__unused__))
> +#endif #ifndef __packed #define __packed __attribute__((packed))
> +#endif
> +
> +#ifndef BIT_ULL
> +#define BIT_ULL(a) (1ULL << (a))
> +#endif
> +
> +#ifndef BIT
> +#define BIT(a) (1ULL << (a))
> +#endif
> +
> +#define FALSE 0
> +#define TRUE 1
> +#define false 0
> +#define true 1
> +
> +#define min(a, b) RTE_MIN(a, b)
> +#define max(a, b) RTE_MAX(a, b)
> +
> +#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0])) #define
> +FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f))) #define MAKEMASK(m, s) ((m)
> +<< (s))
> +
> +#define DEBUGOUT(S) PMD_DRV_LOG_RAW(DEBUG, S) #define
> DEBUGOUT2(S,
> +A...) PMD_DRV_LOG_RAW(DEBUG, S, ##A) #define DEBUGFUNC(F)
> +PMD_DRV_LOG_RAW(DEBUG, F)
> +
> +#define iecm_debug(h, m, s, ...) \
> + do { \
> + if (((m) & (h)->debug_mask)) \
> + PMD_DRV_LOG_RAW(DEBUG, "iecm %02x.%x " s,
> \
> + (h)->bus.device, (h)->bus.func, \
> + ##__VA_ARGS__);
> \
> + } while (0)
> +
> +#define iecm_info(hw, fmt, args...) iecm_debug(hw, IECM_DBG_ALL, fmt,
> +##args) #define iecm_warn(hw, fmt, args...) iecm_debug(hw,
> IECM_DBG_ALL, fmt, ##args)
> +#define iecm_debug_array(hw, type, rowsize, groupsize, buf, len) \
> + do { \
> + struct iecm_hw *hw_l = hw; \
> + u16 len_l = len; \
> + u8 *buf_l = buf; \
> + int i; \
> + for (i = 0; i < len_l; i += 8) \
> + iecm_debug(hw_l, type,
> \
> + "0x%04X 0x%016"PRIx64"\n",
> \
> + i, *((u64 *)((buf_l) + i))); \
> + } while (0)
> +#define iecm_snprintf snprintf
> +#ifndef SNPRINTF
> +#define SNPRINTF iecm_snprintf
> +#endif
> +
> +#define IECM_PCI_REG(reg) rte_read32(reg)
> +#define IECM_PCI_REG_ADDR(a, reg) \
> + ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
> +#define IECM_PCI_REG64(reg) rte_read64(reg)
> +#define IECM_PCI_REG_ADDR64(a, reg) \
> + ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
> +
> +#define iecm_wmb() rte_io_wmb()
> +#define iecm_rmb() rte_io_rmb()
> +#define iecm_mb() rte_io_mb()
> +
> +static inline uint32_t iecm_read_addr(volatile void *addr) {
> + return rte_le_to_cpu_32(IECM_PCI_REG(addr));
> +}
> +
> +static inline uint64_t iecm_read_addr64(volatile void *addr) {
> + return rte_le_to_cpu_64(IECM_PCI_REG64(addr));
> +}
> +
> +#define IECM_PCI_REG_WRITE(reg, value) \
> + rte_write32((rte_cpu_to_le_32(value)), reg)
> +
> +#define IECM_PCI_REG_WRITE64(reg, value) \
> + rte_write64((rte_cpu_to_le_64(value)), reg)
> +
> +#define IECM_READ_REG(hw, reg)
> iecm_read_addr(IECM_PCI_REG_ADDR((hw), (reg)))
> +#define IECM_WRITE_REG(hw, reg, value)
> \
> + IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((hw), (reg)), (value))
> +
> +#define rd32(a, reg) iecm_read_addr(IECM_PCI_REG_ADDR((a), (reg)))
> +#define wr32(a, reg, value) \
> + IECM_PCI_REG_WRITE(IECM_PCI_REG_ADDR((a), (reg)), (value))
> #define
> +div64_long(n, d) ((n) / (d)) #define rd64(a, reg)
> +iecm_read_addr64(IECM_PCI_REG_ADDR64((a), (reg)))
> +
> +#define BITS_PER_BYTE 8
> +
> +/* memory allocation tracking */
> +struct iecm_dma_mem {
> + void *va;
> + u64 pa;
> + u32 size;
> + const void *zone;
> +} __attribute__((packed));
> +
> +struct iecm_virt_mem {
> + void *va;
> + u32 size;
> +} __attribute__((packed));
> +
> +#define iecm_malloc(h, s) rte_zmalloc(NULL, s, 0)
> +#define iecm_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
> +#define iecm_free(h, m) rte_free(m)
> +
> +#define iecm_memset(a, b, c, d) memset((a), (b), (c))
> +#define iecm_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
> +#define iecm_memdup(a, b, c, d) rte_memcpy(iecm_malloc(a, c), b, c)
> +
> +#define CPU_TO_BE16(o) rte_cpu_to_be_16(o) #define CPU_TO_BE32(o)
> +rte_cpu_to_be_32(o) #define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
> #define
> +CPU_TO_LE16(o) rte_cpu_to_le_16(o) #define CPU_TO_LE32(s)
> +rte_cpu_to_le_32(s) #define CPU_TO_LE64(h) rte_cpu_to_le_64(h) #define
> +LE16_TO_CPU(a) rte_le_to_cpu_16(a) #define LE32_TO_CPU(c)
> +rte_le_to_cpu_32(c) #define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
> +
> +#define NTOHS(a) rte_be_to_cpu_16(a)
> +#define NTOHL(a) rte_be_to_cpu_32(a)
> +#define HTONS(a) rte_cpu_to_be_16(a)
> +#define HTONL(a) rte_cpu_to_be_32(a)
> +
> +/* SW spinlock */
> +struct iecm_lock {
> + rte_spinlock_t spinlock;
> +};
> +
> +static inline void
> +iecm_init_lock(struct iecm_lock *sp)
> +{
> + rte_spinlock_init(&sp->spinlock);
> +}
> +
> +static inline void
> +iecm_acquire_lock(struct iecm_lock *sp) {
> + rte_spinlock_lock(&sp->spinlock);
> +}
> +
> +static inline void
> +iecm_release_lock(struct iecm_lock *sp) {
> + rte_spinlock_unlock(&sp->spinlock);
> +}
> +
> +static inline void
> +iecm_destroy_lock(__attribute__((unused)) struct iecm_lock *sp) { }
> +
> +struct iecm_hw;
> +
> +static inline void *
> +iecm_alloc_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
> + struct iecm_dma_mem *mem, u64 size) {
> + const struct rte_memzone *mz = NULL;
> + char z_name[RTE_MEMZONE_NAMESIZE];
> +
> + if (!mem)
> + return NULL;
> +
> + snprintf(z_name, sizeof(z_name), "iecm_dma_%"PRIu64, rte_rand());
> + mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
> + RTE_MEMZONE_IOVA_CONTIG,
> RTE_PGSIZE_4K);
> + if (!mz)
> + return NULL;
> +
> + mem->size = size;
> + mem->va = mz->addr;
> + mem->pa = mz->iova;
> + mem->zone = (const void *)mz;
> + memset(mem->va, 0, size);
> +
> + return mem->va;
> +}
> +
> +static inline void
> +iecm_free_dma_mem(__attribute__((unused)) struct iecm_hw *hw,
> + struct iecm_dma_mem *mem)
> +{
> + rte_memzone_free((const struct rte_memzone *)mem->zone);
> + mem->size = 0;
> + mem->va = NULL;
> + mem->pa = 0;
> +}
> +
> +static inline u8
> +iecm_hweight8(u32 num)
> +{
> + u8 bits = 0;
> + u32 i;
> +
> + for (i = 0; i < 8; i++) {
> + bits += (u8)(num & 0x1);
> + num >>= 1;
> + }
> +
> + return bits;
> +}
> +
> +static inline u8
> +iecm_hweight32(u32 num)
> +{
> + u8 bits = 0;
> + u32 i;
> +
> + for (i = 0; i < 32; i++) {
> + bits += (u8)(num & 0x1);
> + num >>= 1;
> + }
> +
> + return bits;
> +}
> +
> +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) #define DELAY(x)
> +rte_delay_us(x) #define iecm_usec_delay(x) rte_delay_us(x) #define
> +iecm_msec_delay(x, y) rte_delay_us(1000 * (x)) #define udelay(x)
> +DELAY(x) #define msleep(x) DELAY(1000 * (x)) #define usleep_range(min,
> +max) msleep(DIV_ROUND_UP(min, 1000))
> +
> +#ifndef IECM_DBG_TRACE
> +#define IECM_DBG_TRACE BIT_ULL(0)
> +#endif
> +
> +#ifndef DIVIDE_AND_ROUND_UP
> +#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b)) #endif
> +
> +#ifndef IECM_INTEL_VENDOR_ID
> +#define IECM_INTEL_VENDOR_ID 0x8086
> +#endif
> +
> +#ifndef IS_UNICAST_ETHER_ADDR
> +#define IS_UNICAST_ETHER_ADDR(addr) \
> + ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0)) #endif
> +
> +#ifndef IS_MULTICAST_ETHER_ADDR
> +#define IS_MULTICAST_ETHER_ADDR(addr) \
> + ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1)) #endif
> +
> +#ifndef IS_BROADCAST_ETHER_ADDR
> +/* Check whether an address is broadcast. */
> +#define IS_BROADCAST_ETHER_ADDR(addr) \
> + ((bool)((((u16 *)(addr))[0] == ((u16)0xffff)))) #endif
> +
> +#ifndef IS_ZERO_ETHER_ADDR
> +#define IS_ZERO_ETHER_ADDR(addr) \
> + (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
> + ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
> + ((bool)((((u16 *)(addr))[2] == ((u16)0x0))))) #endif
> +
> +#ifndef LIST_HEAD_TYPE
> +#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
> +#endif
> +
> +#ifndef LIST_ENTRY_TYPE
> +#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
> +#endif
> +
> +#ifndef LIST_FOR_EACH_ENTRY_SAFE
> +#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list)
> \
> + LIST_FOREACH(pos, head, list)
> +
> +#endif
> +
> +#ifndef LIST_FOR_EACH_ENTRY
> +#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
> + LIST_FOREACH(pos, head, list)
> +
> +#endif
> +
> +#endif /* _IECM_OSDEP_H_ */
> diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h new
> file mode 100644 index 0000000000..906aae8463
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_logs.h
> @@ -0,0 +1,38 @@
License description is missed in this new file.
> +#ifndef _IDPF_LOGS_H_
> +#define _IDPF_LOGS_H_
> +
> +#include <rte_log.h>
> +
> +extern int idpf_logtype_init;
> +extern int idpf_logtype_driver;
> +
> +#define PMD_INIT_LOG(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
> + "%s(): " fmt "\n", __func__, ##args)
> +
> +#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
> +
> +#define PMD_DRV_LOG_RAW(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
> + "%s(): " fmt "\n", __func__, ##args)
> +
> +#define PMD_DRV_LOG(level, fmt, args...) \
> + PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
> +
> +#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
> +
> +#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
> +#define PMD_RX_LOG(level, fmt, args...) \
> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args) #else
> +#define PMD_RX_LOG(level, fmt, args...) do { } while (0) #endif
> +
> +#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
> +#define PMD_TX_LOG(level, fmt, args...) \
> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args) #else
> +#define PMD_TX_LOG(level, fmt, args...) do { } while (0) #endif
> +
> +#endif /* _IDPF_LOGS_H_ */
> --
> 2.25.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 01/14] net/idpf/base: introduce base code
2022-10-03 13:20 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 21:20
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v2 01/14] net/idpf/base: introduce base code
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Introduce base code for IDPF (Infrastructure Data Path Function) PMD.
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> I'm OK to import base driver in one patch, but it should not be
> the first patch with dead code which is not actually built
> just after the patch.
>
> The first patch should add a driver stub which is built.
> Subsequent patches should add base driver and functionality
> to the ethdev driver itself.
Sure, thanks for the advice!
We will do some refactor work and move the base driver at common folder.
That would look better in the coming version. Thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation
2022-10-03 13:20 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 21:20
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v2 02/14] net/idpf/base: add logs and OS specific
> implementation
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add PMD logs.
> > Add some MACRO definations and small functions which are specific
> > for DPDK.
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> The patch adds dead code as well. Added headers are used
> nowhere.
>
> One more question is what's the point to have two things in one
> patch? Typically such descriptions suggest that it could be
> separate patches in a right order.
Sure, will improve this in the coming version patchset. Thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 03/14] net/idpf: add support for device initialization
2022-10-03 13:44 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 21:44
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Li, Xiaoyun
> <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v2 03/14] net/idpf: add support for device
> initialization
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Support device init and the following dev ops:
> > - dev_configure
> > - dev_start
> > - dev_stop
> > - dev_close
>
> A bit strange set from my point of view since you can't start
> without setup queues. Each patch should be testable.
> It should be no dead code. It should be possible to stop after
> the patch, build, run, for example, testpmd and probe,
> configure, start, stop, close, unplug (depeneding on stage in
> the patch series).
>
> The patch is really big. It definitely worse to split
> configre/close and start/stop (and reorder to have start
> after queues setup supported).
Well, this patch mainly implemented the device initiation, as well
as adding the framework/skeleton of the config/start/stop/close
ops. As for the dev start ops, we decide to firstly add the vports
setup at this stage in this patch and later add other setups like
queues and queue irqs. So in this way, it may be reasonable that
the device could still start even without the queues setup.
And yes, this PMD contains too much code with so many INITs.
We will try to split them in the coming versions. Thanks!
>
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> > ---
> > drivers/net/idpf/idpf_ethdev.c | 810
> +++++++++++++++++++++++++++++++++
> > drivers/net/idpf/idpf_ethdev.h | 229 ++++++++++
> > drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++++++
> > drivers/net/idpf/meson.build | 18 +
> > drivers/net/idpf/version.map | 3 +
> > drivers/net/meson.build | 1 +
> > 6 files changed, 1556 insertions(+)
> > create mode 100644 drivers/net/idpf/idpf_ethdev.c
> > create mode 100644 drivers/net/idpf/idpf_ethdev.h
> > create mode 100644 drivers/net/idpf/idpf_vchnl.c
> > create mode 100644 drivers/net/idpf/meson.build
> > create mode 100644 drivers/net/idpf/version.map
> >
> > diff --git a/drivers/net/idpf/idpf_ethdev.c
> b/drivers/net/idpf/idpf_ethdev.c
> > new file mode 100644
> > index 0000000000..f0452de09e
> > --- /dev/null
> > +++ b/drivers/net/idpf/idpf_ethdev.c
> > @@ -0,0 +1,810 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2022 Intel Corporation
> > + */
> > +
> > +#include <rte_atomic.h>
> > +#include <rte_eal.h>
> > +#include <rte_ether.h>
> > +#include <rte_malloc.h>
> > +#include <rte_memzone.h>
> > +#include <rte_dev.h>
> > +
> > +#include "idpf_ethdev.h"
> > +
> > +#define IDPF_VPORT "vport"
> > +
> > +struct idpf_adapter_list adapter_list;
> > +bool adapter_list_init;
> > +
> > +uint64_t idpf_timestamp_dynflag;
> > +
> > +static const char * const idpf_valid_args[] = {
> > + IDPF_VPORT,
> > + NULL
> > +};
> > +
> > +static int idpf_dev_configure(struct rte_eth_dev *dev);
> > +static int idpf_dev_start(struct rte_eth_dev *dev);
> > +static int idpf_dev_stop(struct rte_eth_dev *dev);
> > +static int idpf_dev_close(struct rte_eth_dev *dev);
> > +
> > +static const struct eth_dev_ops idpf_eth_dev_ops = {
> > + .dev_configure = idpf_dev_configure,
> > + .dev_start = idpf_dev_start,
> > + .dev_stop = idpf_dev_stop,
> > + .dev_close = idpf_dev_close,
> > +};
> > +
> > +static int
> > +idpf_init_vport_req_info(struct rte_eth_dev *dev)
> > +{
> > + struct idpf_vport *vport = dev->data->dev_private;
> > + struct idpf_adapter *adapter = vport->adapter;
> > + struct virtchnl2_create_vport *vport_info;
> > + uint16_t idx = adapter->next_vport_idx;
> > +
> > + if (!adapter->vport_req_info[idx]) {
>
> DPDK coding style requires explicit comparison vs NULL [1].
> It is not applicable to base driver, but PMD itself should
> follow it. There is really huge number of places to fix.
>
> It is applicable to comparision integers vs 0 as well.
Will fix these in the coming versions. Thanks!
>
> [1] https://doc.dpdk.org/guides/contributing/coding_style.html#null-
> pointers
>
> [snip]
>
> > +static int
> > +idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
> > +{
>
> I'm really confusing that the configure as empty.
> It must validate configuration provided via
> rte_eth_dev_configure() and reject if something is not
> supported if ethdev does not the job.
Well, the initiation of some resources like vports are not put at this stage.
It is flexible to add the framework/skeleton of the config/start/stop/close
ops. Looks that it ok to implement like this... Thanks!
>
> > + return 0;
> > +}
>
> [snip]
>
> > diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> > new file mode 100644
> > index 0000000000..7d776d3a15
> > --- /dev/null
> > +++ b/drivers/net/idpf/meson.build
> > @@ -0,0 +1,18 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(c) 2022 Intel Corporation
> > +
> > +if is_windows
> > + build = false
> > + reason = 'not supported on Windows'
> > + subdir_done()
> > +endif
> > +
> > +subdir('base')
> > +objs = [base_objs]
> > +
> > +sources = files(
> > + 'idpf_ethdev.c',
> > + 'idpf_vchnl.c',
>
> TAB vs spaces? Or is it just mail glitch?
Will double check for this, thanks!
>
> > +)
> > +
> > +includes += include_directories('base')
> > \ No newline at end of file
> > diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
> > new file mode 100644
> > index 0000000000..b7da224860
> > --- /dev/null
> > +++ b/drivers/net/idpf/version.map
> > @@ -0,0 +1,3 @@
> > +DPDK_22 {
> > + local: *;
> > +};
> > \ No newline at end of file
>
> It must be an empty line at the end of file.
Newline at EOF is not required for DPDK code.
This warning may be added by mistake.
Will double check for this, thanks!
>
> > diff --git a/drivers/net/meson.build b/drivers/net/meson.build
> > index e35652fe63..8faf8120c2 100644
> > --- a/drivers/net/meson.build
> > +++ b/drivers/net/meson.build
> > @@ -28,6 +28,7 @@ drivers = [
> > 'i40e',
> > 'iavf',
> > 'ice',
> > + 'idpf',
>
> TAB vs spaces? Or is it just mail glitch?
Will double check for this, thanks!
>
> > 'igc',
> > 'ionic',
> > 'ipn3ke',
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 04/14] net/idpf: add support for queue operations
2022-10-03 13:47 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 21:48
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Li, Xiaoyun
> <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v2 04/14] net/idpf: add support for queue operations
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add support for queue operations:
> > - rx_queue_start
> > - rx_queue_stop
> > - tx_queue_start
> > - tx_queue_stop
> > - rx_queue_setup
> > - rx_queue_release
> > - tx_queue_setup
> > - tx_queue_release
>
> The patch is huge. Please, factor out deferred queues
> start/stop (may be even separate for Rx and Tx) into
> separate patch.
Okay... Will try to do this. Thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 06/14] net/idpf: add support for packet type get
2022-10-03 13:58 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Wu, Wenjun1
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 21:59
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Wu,
> Wenjun1 <wenjun1.wu@intel.com>
> Subject: Re: [PATCH v2 06/14] net/idpf: add support for packet type get
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add dev ops dev_supported_ptypes_get.
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> > index fe044a80c9..1c5c4688cc 100644
> > --- a/drivers/net/idpf/idpf_rxtx.c
> > +++ b/drivers/net/idpf/idpf_rxtx.c
> > @@ -8,6 +8,25 @@
> > #include "idpf_ethdev.h"
> > #include "idpf_rxtx.h"
> >
> > +const uint32_t *
> > +idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev
> __rte_unused)
> > +{
> > + static const uint32_t ptypes[] = {
> > + RTE_PTYPE_L2_ETHER,
> > + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
> > + RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
> > + RTE_PTYPE_L4_FRAG,
> > + RTE_PTYPE_L4_NONFRAG,
> > + RTE_PTYPE_L4_UDP,
> > + RTE_PTYPE_L4_TCP,
> > + RTE_PTYPE_L4_SCTP,
> > + RTE_PTYPE_L4_ICMP,
> > + RTE_PTYPE_UNKNOWN
> > + };
>
> How am I supported to verify that these packet types are really
> recognized by the driver? The patch should include the part
> of Rx burst callback which does recognition and fill in
> corresponding data in mbuf.
You can verify the PTYPE via sending & receiving certain packets.
After setting the cmd verbose, the corresponding PTYPE may show
up in the testpmd. You can then check if the PTYPE is exactly what
you just sent.
The ptype_tbl will be filled when func idpf_get_pkt_type is called.
And then the corresponding part will be filled in Rx burst phase.
This part patch will also be improved to make it more clear. Thanks!
>
> > +
> > + return ptypes;
> > +}
> > +
> > static inline int
> > check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
> > {
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-03 14:02 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 22:03
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Li, Xiaoyun
> <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx
> datapath
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add basic RX & TX support in split queue mode and single queue mode.
>
> RX -> Rx, TX -> Tx
>
> > Split queue mode is selected by default.
>
> What is split queue mode? Where is it defined/described?
>
> The patch is very big again. Please, start from really basic Rx
> and Tx functionality and then add offloads one by one.
> Otherwise it is really hard to review everything at once.
Sure, will split the basic Rx/Tx and the offloads, and also add
doc to describe the queue modes. Thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 10/14] net/idpf: add support for mtu configuration
2022-10-03 14:12 ` Andrew Rybchenko
@ 2022-10-14 9:18 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:18 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 22:12
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v2 10/14] net/idpf: add support for mtu configuration
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add dev ops mtu_set.
>
> It would be useful to mention in the description that
> changing MTU is not supported when device is started.
Okay... There will be a feature list to announce the supported features.
And the supported features may be added gradually along with each commit.
That would be clear enough to track. Thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-03 14:20 ` Andrew Rybchenko
@ 2022-10-14 9:19 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:19 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Wu, Wenjun1, Thomas Monjalon
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 22:21
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Wu,
> Wenjun1 <wenjun1.wu@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [PATCH v2 13/14] net/idpf: add AVX512 data path for single
> queue model
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add support of AVX512 vector data path for single queue model.
> >
> > Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > +static __rte_always_inline void
> > +idpf_singleq_rearm(struct idpf_rx_queue *rxq)
> > +{
>
> [snip]
>
> > + const __m512i iova_offsets = _mm512_set1_epi64(offsetof
> > + (struct rte_mbuf,
> buf_iova));
> > + const __m512i headroom =
> _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
> > +
> > +#ifndef RTE_LIBRTE_IDPF_16BYTE_RX_DESC
>
> As far as I remember, support for such build time options is
> deprecated in DPDK and new options are not accepted.
> Cc Thomas to correct me if I'm wrong.
Yes, we will also remove this MACRO in the coming version. Thanks!
>
> > + /* to shuffle the addresses to correct slots. Values 4-7 will
> contain
> > + * zeros, so use 7 for a zero-value.
> > + */
> > + const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7,
> 2, 0);
> > +#else
> > + const __m512i permute_idx = _mm512_set_epi64(7, 3, 6, 2, 5, 1,
> 4, 0);
> > +#endif
>
> [snip]
>
> > +#ifdef IDPF_RX_PTYPE_OFFLOAD
This one is only a local MACRO defined in this file. So it may have no impact.
Thanks!
>
> Same here.
>
> > + /**
> > + * to get packet types, shift 64-bit values down 30 bits
> > + * and so ptype is in lower 8-bits in each
> > + */
> > + const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7,
> 16);
> > + const __m256i ptypes6_7 =
> _mm512_extracti64x4_epi64(ptypes4_7, 1);
> > + const __m256i ptypes4_5 =
> _mm512_extracti64x4_epi64(ptypes4_7, 0);
> > + const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7,
> 16);
> > + const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7,
> 0);
> > + const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5,
> 16);
> > + const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5,
> 0);
> > +
> > + const __m512i ptype4_7 = _mm512_set_epi32
> > + (0, 0, 0, type_table[ptype7],
> > + 0, 0, 0, type_table[ptype6],
> > + 0, 0, 0, type_table[ptype5],
> > + 0, 0, 0, type_table[ptype4]);
> > + mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7,
> ptype4_7);
> > +#endif
>
> [snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 14/14] net/idpf: add support for timestamp offload
2022-10-03 14:22 ` Andrew Rybchenko
@ 2022-10-14 9:19 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-14 9:19 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W, Qiao, Wenjing
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 22:22
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>; Qiao,
> Wenjing <wenjing.qiao@intel.com>
> Subject: Re: [PATCH v2 14/14] net/idpf: add support for timestamp
> offload
>
> On 9/5/22 13:58, Junfeng Guo wrote:
> > Add support for timestamp offload.
> >
> > Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > +/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
> > +static inline uint64_t
> > +idpf_tstamp_convert_32b_64b(struct iecm_hw *hw, struct
> idpf_adapter *ad,
> > + uint32_t flag, uint32_t in_timestamp)
> > +{
> > +/* TODO: timestamp for ACC */
> > +#ifdef RTE_ARCH_ARM64
> > + return 0;
> > +#endif /* RTE_ARCH_ARM64 */
> > +
> > +#ifdef RTE_ARCH_X86_64
> > + const uint64_t mask = 0xFFFFFFFF;
> > + uint32_t hi, lo, lo2, delta;
> > + uint64_t ns;
> > +
> > + if (flag) {
> > + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0,
> PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> > + IECM_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0,
> PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
> > + PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
> > + lo = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
> > + hi = IECM_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
> > + /*
> > + * On typical system, the delta between lo and lo2 is
> ~1000ns,
> > + * so 10000 seems a large-enough but not overly-big
> guard band.
> > + */
> > + if (lo > (UINT32_MAX -
> IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
> > + lo2 = IECM_READ_REG(hw,
> PF_GLTSYN_SHTIME_L_0);
> > + else
> > + lo2 = lo;
> > +
> > + if (lo2 < lo) {
> > + lo = IECM_READ_REG(hw,
> PF_GLTSYN_SHTIME_L_0);
> > + hi = IECM_READ_REG(hw,
> PF_GLTSYN_SHTIME_H_0);
> > + }
> > +
> > + ad->time_hw = ((uint64_t)hi << 32) | lo;
> > + }
> > +
> > + delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
> > + if (delta > (mask / 2)) {
> > + delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
> > + ns = ad->time_hw - delta;
> > + } else {
> > + ns = ad->time_hw + delta;
> > + }
> > +
> > + return ns;
> > +#endif /* RTE_ARCH_X86_64 */
>
> Conditional compilation is rather strange above.
> Will it break build on some architectures?
> Non-x86-64 and non-ARM64.
Actually the timestamp feature currently is only supported on x86_64
architecture. On the rest conditions will just return 0. We will update
this part in the coming version. Thanks!
>
> > +}
> >
> > +#endif /* _IDPF_RXTX_H_ */
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v2 00/14] add support for idpf PMD in DPDK
2022-10-03 14:36 ` Andrew Rybchenko
@ 2022-10-18 11:09 ` Guo, Junfeng
0 siblings, 0 replies; 376+ messages in thread
From: Guo, Junfeng @ 2022-10-18 11:09 UTC (permalink / raw)
To: Andrew Rybchenko, Zhang, Qi Z, Wu, Jingjing, Xing, Beilei
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, October 3, 2022 22:36
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v2 00/14] add support for idpf PMD in DPDK
>
> On 10/3/22 16:31, Andrew Rybchenko wrote:
> > On 9/5/22 13:58, Junfeng Guo wrote:
> >> This patchset introduced the idpf (Infrastructure Data Path Function)
> >> PMD [*EXPERIMENTAL*] in DPDK for Intel Device ID of 0x1452.
> >>
> >> v2:
> >> fixed some coding style issues and did some refactors.
> >
> > The patch series should:
> > - add entry in MAINTAINERS
> > - add driver documentation to doc/guides/nics/idpf.rst
> > - advertise the driver in release notes
> > - when features are added it should be added to
> > doc/guides/nics/features/idpf.ini
Sure, thanks for your advice!
We will improve the commit order and items in the coming versions.
Currently we are still trying to improve the description related files.
Please also help review for the coming versions, thanks a lot!
>
> I'd really like to add one more point.
> There is a long road for the driver to be accepted since
> many changes are required, but it is definitely doable.
>
> More important to get replies on generic questions.
> What is it? What is the HW? Is it released?
> Where I can read about it?
Well, this is the new ``idpf`` net driver for Intel® IPU Ethernet
ES2000 Series. The spec has not been released at this point.
Could you help review the coming version patch set, thanks!
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 00/15] add support for idpf PMD in DPDK
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (14 preceding siblings ...)
2022-10-03 13:31 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Andrew Rybchenko
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 01/15] common/idpf: introduce common library Junfeng Guo
` (14 more replies)
15 siblings, 15 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=a, Size: 4885 bytes --]
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU Ethernet ES2000 Series (Device ID: 0x1452).
v2:
fixed some coding style issues and did some refactors.
v3:
fixed some coding style issues and did some refactors.
Junfeng Guo (15):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for link status update
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 365 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 41 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2398 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 870 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1443 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15368 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 01/15] common/idpf: introduce common library
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 02/15] net/idpf: add support for device initialization Junfeng Guo
` (13 subsequent siblings)
14 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definations and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 365 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 41 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8494 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..df8222d42e
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("idpf_set_mac_type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("idpf_set_mac_type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..7256a3e574
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ (u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1)
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..2a8d8f9295
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+#define IDPF_DEV_ID_CPF 0x1453
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..da0d3b0ffb
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap belwo comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..fd931869a0
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..648afcd4e5
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,365 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum idpf_status idpf_status;
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0]))
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3214c5e068
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p) (_p);
+#define UNREFERENCED_2PARAMETER(_p, _q) (_p); (_q);
+#define UNREFERENCED_3PARAMETER(_p, _q, _r) (_p); (_q); (_r);
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s) (_p); (_q); (_r); (_s);
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t) (_p); (_q); (_r); (_s); (_t);
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..3f1f421ef1
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-but-set-variable']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..bb7b2daac0
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000 /* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000)) /* 2k Rx buffer queues */
+
+#define VDEV_QTX_TAIL_START 0x1100000 /* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000 /* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000)) /* 2k Tx completion queues */
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000) /* Begin at offset of 33MB + 4k to accomdate CTL01 register */
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..dcb4141ff2
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..03481f1545
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids upto 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is trasmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..62884fb4ca
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ | 16| 0|
+ ----------------------------------------------------------------
+ | RSV | Buffer ID |
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ | 0|
+ ----------------------------------------------------------------
+ | Rx packet buffer adresss |
+ ----------------------------------------------------------------
+ | Rx header buffer adresss |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | RSV |
+ ----------------------------------------------------------------
+ | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..902f63bd51
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length caculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 02/15] net/idpf: add support for device initialization
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 01/15] common/idpf: introduce common library Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 03/15] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (12 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li, Xiao Wang
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=a, Size: 45081 bytes --]
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/net/idpf/idpf_ethdev.c | 753 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 198 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1590 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2bd4a55f1b..e2d9c2ac50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -758,6 +758,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..9900e23364
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
\ No newline at end of file
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..417d6b690e
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+50/100/200 Gbps Intel® IPU Ethernet ES2000 Series Network Adapters.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 2da8bc9661..16e7f39ce9 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -236,6 +236,11 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD.**
+
+ Added the new ``idpf`` net driver for Intel® IPU Ethernet ES2000 Series.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..e73a30a00f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,753 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to creat vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..d00ee12364
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,198 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 03/15] net/idpf: add queue setup and release in single queue model
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 01/15] common/idpf: introduce common library Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 02/15] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 04/15] net/idpf: add queue setup and release in split " Junfeng Guo
` (11 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=a, Size: 34377 bytes --]
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 463 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 249 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 990 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 9900e23364..7cfcb60bd0 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 417d6b690e..af7810834a 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (defalut ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (defalut ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index e73a30a00f..c0a6b3e90d 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -32,6 +37,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static int
@@ -60,6 +69,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -89,6 +110,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -128,6 +151,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -291,6 +320,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -312,6 +360,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index d00ee12364..d90bcfa63a 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -114,6 +116,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -191,6 +196,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..9fef467990
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,463 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (!txq->sw_nb_desc) {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_rx_queue *rxq;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t len;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..02a55e0658 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,254 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 04/15] net/idpf: add queue setup and release in split queue model
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (2 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 03/15] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 05/15] net/idpf: add support for queue start and stop Junfeng Guo
` (10 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 485 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 644 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c0a6b3e90d..21520fa648 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -69,13 +69,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -113,7 +125,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -136,6 +150,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index d90bcfa63a..9f8fcba4ca 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9fef467990..07ee8f1e14 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -97,7 +97,10 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
return;
}
- if (!txq->sw_nb_desc) {
+ if (txq->sw_nb_desc) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
}
@@ -125,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -147,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -179,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -216,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint16_t qid;
+ uint16_t len;
+ uint64_t offloads;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -335,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -447,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 02a55e0658..87a024b6db 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,10 +481,10 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, err, i;
+ int size, err, i, j;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -520,7 +520,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -550,7 +589,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -581,7 +620,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -612,7 +683,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int size, err, i;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -642,7 +713,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -700,7 +793,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 05/15] net/idpf: add support for queue start and stop
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (3 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 04/15] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 06/15] net/idpf: add support for device information get Junfeng Guo
` (9 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 7cfcb60bd0..ffcb7caa0f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 21520fa648..b4923faa5f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -37,6 +37,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -190,6 +194,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -199,6 +236,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -207,6 +249,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -223,6 +267,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9f8fcba4ca..c03b051f87 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -211,6 +211,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 07ee8f1e14..c525e9b4c9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 87a024b6db..da48eef29d 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -829,6 +829,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 06/15] net/idpf: add support for device information get
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (4 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 05/15] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 07/15] net/idpf: add support for packet type get Junfeng Guo
` (8 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index b4923faa5f..7ae52a68b2 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
@@ -45,8 +47,56 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 07/15] net/idpf: add support for packet type get
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (5 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 06/15] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 08/15] net/idpf: add support for link status update Junfeng Guo
` (7 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ffcb7caa0f..f1849cd821 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7ae52a68b2..10ae5f14f1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,7 @@ static int idpf_dev_info_get(struct rte_eth_dev *dev,
static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -631,6 +632,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c03b051f87..64da6fd2a5 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -204,6 +204,7 @@ _atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -215,6 +216,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index c525e9b4c9..846e1cdfc6 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index da48eef29d..72c7d303a9 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1003,3 +1212,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 08/15] net/idpf: add support for link status update
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (6 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 07/15] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 09/15] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 21 +++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 6 ++++++
2 files changed, 27 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 10ae5f14f1..ff0833b462 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -34,6 +34,26 @@ static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
@@ -49,6 +69,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 64da6fd2a5..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -110,6 +110,10 @@ struct idpf_vport {
/* Chunk info */
struct idpf_chunks_info chunks_info;
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
uint16_t devarg_id;
bool stopped;
struct virtchnl2_vport_stats eth_stats_offset;
@@ -202,6 +206,8 @@ _atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
}
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_get_pkt_type(struct idpf_adapter *adapter);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 09/15] net/idpf: add support for basic Rx/Tx datapath
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (7 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 08/15] net/idpf: add support for link status update Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 10/15] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 671 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 691 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ff0833b462..c6a2c85f17 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 846e1cdfc6..b2c2dca0a8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,674 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (int i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (int i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ingnore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ return;
+ } else {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ return;
+ }
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 10/15] net/idpf: add support for Rx/Tx offloading
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (8 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 09/15] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 11/15] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 267 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f1849cd821..23953c384d 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6a2c85f17..d09de4075c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index b2c2dca0a8..ff3b710f4d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1342,9 +1383,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1419,6 +1462,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1495,6 +1543,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1503,11 +1594,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1537,8 +1631,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1565,6 +1680,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1579,6 +1696,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1612,6 +1771,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1681,10 +1841,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1740,6 +1903,31 @@ idpf_xmit_cleanup(struct idpf_tx_queue *txq)
return 0;
}
+/* set TSO context descriptor
+ * support IP -> L4 and IP -> IP -> L4
+ */
+static inline uint64_t
+idpf_set_tso_ctx(struct rte_mbuf *mbuf, union idpf_tx_offload tx_offload)
+{
+ uint64_t ctx_desc = 0;
+ uint32_t cd_cmd, hdr_len, cd_tso_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return ctx_desc;
+ }
+
+ hdr_len = tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len;
+
+ cd_cmd = IDPF_TX_CTX_DESC_TSO;
+ cd_tso_len = mbuf->pkt_len - hdr_len;
+ ctx_desc |= ((uint64_t)cd_cmd << IDPF_TXD_CTX_QW1_CMD_S) |
+ ((uint64_t)cd_tso_len << IDPF_TXD_CTX_QW1_TSO_LEN_S) |
+ ((uint64_t)mbuf->tso_segsz << IDPF_TXD_CTX_QW1_MSS_S);
+
+ return ctx_desc;
+}
+
/* TX function */
uint16_t
idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
@@ -1747,14 +1935,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1781,11 +1972,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1813,6 +2012,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1895,11 +2116,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 11/15] net/idpf: add support for RSS
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (9 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 10/15] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 12/15] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d09de4075c..d89a3f2312 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ff3b710f4d..00d941b3ba 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1464,6 +1490,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 72c7d303a9..c5290d8050 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 12/15] net/idpf: add support for MTU configuration
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (10 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 11/15] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 13/15] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 23953c384d..0123865fd3 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d89a3f2312..cb6051be9c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 13/15] net/idpf: add support for write back based on ITR expire
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (11 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 12/15] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 14/15] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 15/15] net/idpf: add support for timestamp offload Junfeng Guo
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cb6051be9c..4fe442135c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are writen back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index c5290d8050..c4aa744953 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1134,6 +1134,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 14/15] net/idpf: add AVX512 data path for single queue model
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (12 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 13/15] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 15/15] net/idpf: add support for timestamp offload Junfeng Guo
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 139 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 870 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1169 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index af7810834a..2e9628a2d0 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 00d941b3ba..730f87fd4a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2170,15 +2172,110 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
return;
}
@@ -2188,12 +2285,54 @@ void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
return;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..2ff1e8ac14
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,870 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_commit = nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v3 15/15] net/idpf: add support for timestamp offload
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
` (13 preceding siblings ...)
2022-10-18 11:12 ` [PATCH v3 14/15] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-18 11:12 ` Junfeng Guo
14 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-18 11:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 58 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 156 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 0123865fd3..37f02c6781 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4fe442135c..10c95d8754 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 730f87fd4a..8756d60cd0 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(
+ &idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1063,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1412,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1421,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1493,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1796,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1877,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 00/14] add support for idpf PMD in DPDK
2022-10-18 11:12 ` [PATCH v3 01/15] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU Ethernet ES2000 Series (Device ID: 0x1452).
v2:
fixed some coding style issues and did some refactors.
v3:
fixed some coding style issues and did some refactors.
v4:
fixed some coding style issues and build errors.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2398 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1443 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15385 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 01/14] common/idpf: introduce common library
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8510 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..20706bc2c6
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,375 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum idpf_status idpf_status;
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..3f1f421ef1
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-but-set-variable']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..f2a1039863
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer adresss |
+ * ----------------------------------------------------------------
+ * | Rx header buffer adresss |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer adresss |
+ * ----------------------------------------------------------------
+ * | Rx header buffer adresss |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 02/14] net/idpf: add support for device initialization
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 204 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1617 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2bd4a55f1b..e2d9c2ac50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -758,6 +758,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..cc56db6e70
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+50/100/200 Gbps Intel® IPU Ethernet ES2000 Series Network Adapters.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 2da8bc9661..6697b61848 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -236,6 +236,11 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® IPU Ethernet ES2000 Series.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..77824d5f7f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 03/14] net/idpf: add queue setup and release in single queue model
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 249 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 992 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index cc56db6e70..84b21e7be0 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 77824d5f7f..8b7170f49e 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -197,6 +202,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..02a55e0658 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,254 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 04/14] net/idpf: add queue setup and release in split queue model
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8b7170f49e..270ab7b6de 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 02a55e0658..87a024b6db 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,10 +481,10 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, err, i;
+ int size, err, i, j;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -520,7 +520,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -550,7 +589,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -581,7 +620,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -612,7 +683,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int size, err, i;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -642,7 +713,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -700,7 +793,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 05/14] net/idpf: add support for queue start and stop
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 270ab7b6de..b625868ceb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 87a024b6db..da48eef29d 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -829,6 +829,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 06/14] net/idpf: add support for device information get
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 07/14] net/idpf: add support for packet type get
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index b625868ceb..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -210,6 +210,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -221,6 +222,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index da48eef29d..72c7d303a9 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1003,3 +1212,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 267 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..a56db1950f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1741,6 +1904,31 @@ idpf_xmit_cleanup(struct idpf_tx_queue *txq)
return 0;
}
+/* set TSO context descriptor
+ * support IP -> L4 and IP -> IP -> L4
+ */
+static inline uint64_t
+idpf_set_tso_ctx(struct rte_mbuf *mbuf, union idpf_tx_offload tx_offload)
+{
+ uint64_t ctx_desc = 0;
+ uint32_t cd_cmd, hdr_len, cd_tso_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return ctx_desc;
+ }
+
+ hdr_len = tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len;
+
+ cd_cmd = IDPF_TX_CTX_DESC_TSO;
+ cd_tso_len = mbuf->pkt_len - hdr_len;
+ ctx_desc |= ((uint64_t)cd_cmd << IDPF_TXD_CTX_QW1_CMD_S) |
+ ((uint64_t)cd_tso_len << IDPF_TXD_CTX_QW1_TSO_LEN_S) |
+ ((uint64_t)mbuf->tso_segsz << IDPF_TXD_CTX_QW1_MSS_S);
+
+ return ctx_desc;
+}
+
/* TX function */
uint16_t
idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
@@ -1748,14 +1936,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1973,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +2013,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2117,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 10/14] net/idpf: add support for RSS
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a56db1950f..88389250be 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 72c7d303a9..c5290d8050 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 11/14] net/idpf: add support for MTU configuration
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 12/14] net/idpf: add support for write back based on ITR expire
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index c5290d8050..c4aa744953 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1134,6 +1134,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 144 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1175 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 84b21e7be0..8f81894e27 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 88389250be..1e3cea30a8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2171,26 +2173,168 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+#ifdef RTE_ARCH_X86
+ else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v4 14/14] net/idpf: add support for timestamp offload
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-19 10:37 ` [PATCH v4 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-19 10:37 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 10:37 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 1e3cea30a8..723b3b974b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 00/14] add support for idpf PMD in DPDK
2022-10-19 10:37 ` [PATCH v4 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU Ethernet ES2000 Series (Device ID: 0x1452).
v2:
fixed some coding style issues and did some refactors.
v3:
fixed some coding style issues and did some refactors.
v4:
fixed some coding style issues and build errors.
v5:
fixed typo.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2400 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1443 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15387 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 01/14] common/idpf: introduce common library
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 15 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8510 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..20706bc2c6
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,375 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum idpf_status idpf_status;
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..3f1f421ef1
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-but-set-variable']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 02/14] net/idpf: add support for device initialization
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 204 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1617 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2bd4a55f1b..e2d9c2ac50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -758,6 +758,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..cc56db6e70
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+50/100/200 Gbps Intel® IPU Ethernet ES2000 Series Network Adapters.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 2da8bc9661..6697b61848 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -236,6 +236,11 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® IPU Ethernet ES2000 Series.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..77824d5f7f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 03/14] net/idpf: add queue setup and release in single queue model
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 249 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 992 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index cc56db6e70..84b21e7be0 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 77824d5f7f..8b7170f49e 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -197,6 +202,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..02a55e0658 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,254 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 04/14] net/idpf: add queue setup and release in split queue model
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8b7170f49e..270ab7b6de 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 02a55e0658..87a024b6db 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,10 +481,10 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, err, i;
+ int size, err, i, j;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -520,7 +520,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -550,7 +589,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -581,7 +620,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -612,7 +683,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int size, err, i;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -642,7 +713,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -700,7 +793,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 05/14] net/idpf: add support for queue start and stop
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 270ab7b6de..b625868ceb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 87a024b6db..da48eef29d 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -829,6 +829,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 06/14] net/idpf: add support for device information get
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 07/14] net/idpf: add support for packet type get
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index b625868ceb..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -210,6 +210,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -221,6 +222,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index da48eef29d..72c7d303a9 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1003,3 +1212,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 267 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..a56db1950f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1741,6 +1904,31 @@ idpf_xmit_cleanup(struct idpf_tx_queue *txq)
return 0;
}
+/* set TSO context descriptor
+ * support IP -> L4 and IP -> IP -> L4
+ */
+static inline uint64_t
+idpf_set_tso_ctx(struct rte_mbuf *mbuf, union idpf_tx_offload tx_offload)
+{
+ uint64_t ctx_desc = 0;
+ uint32_t cd_cmd, hdr_len, cd_tso_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return ctx_desc;
+ }
+
+ hdr_len = tx_offload.l2_len + tx_offload.l3_len + tx_offload.l4_len;
+
+ cd_cmd = IDPF_TX_CTX_DESC_TSO;
+ cd_tso_len = mbuf->pkt_len - hdr_len;
+ ctx_desc |= ((uint64_t)cd_cmd << IDPF_TXD_CTX_QW1_CMD_S) |
+ ((uint64_t)cd_tso_len << IDPF_TXD_CTX_QW1_TSO_LEN_S) |
+ ((uint64_t)mbuf->tso_segsz << IDPF_TXD_CTX_QW1_MSS_S);
+
+ return ctx_desc;
+}
+
/* TX function */
uint16_t
idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
@@ -1748,14 +1936,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1973,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +2013,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2117,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 10/14] net/idpf: add support for RSS
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a56db1950f..88389250be 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 72c7d303a9..c5290d8050 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 11/14] net/idpf: add support for MTU configuration
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 12/14] net/idpf: add support for write back based on ITR expire
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, junfeng.guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index c5290d8050..c4aa744953 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1134,6 +1134,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 146 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1177 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 84b21e7be0..8f81894e27 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 88389250be..6bfd3ae583 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2171,26 +2173,170 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v5 14/14] net/idpf: add support for timestamp offload
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-19 11:03 ` [PATCH v5 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-19 11:03 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 11:03 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, junfeng.guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6bfd3ae583..b13a2c3494 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 00/14] add support for idpf PMD in DPDK
2022-10-19 11:03 ` [PATCH v5 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU Ethernet ES2000 Series (Device ID: 0x1452).
v2:
fixed some coding style issues and did some refactors.
v3:
fixed some coding style issues and did some refactors.
v4:
fixed some coding style issues and build errors.
v5:
fixed typo.
v6:
fixed build errors.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2375 +++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1443 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15361 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 01/14] common/idpf: introduce common library
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8509 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..20706bc2c6
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,375 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum idpf_status idpf_status;
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..183587b51a
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 02/14] net/idpf: add support for device initialization
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 204 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1617 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2bd4a55f1b..e2d9c2ac50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -758,6 +758,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..cc56db6e70
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+50/100/200 Gbps Intel® IPU Ethernet ES2000 Series Network Adapters.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 2da8bc9661..6697b61848 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -236,6 +236,11 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® IPU Ethernet ES2000 Series.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..77824d5f7f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 03/14] net/idpf: add queue setup and release in single queue model
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 249 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 992 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index cc56db6e70..84b21e7be0 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 77824d5f7f..8b7170f49e 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -197,6 +202,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..02a55e0658 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,254 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, err, i;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 04/14] net/idpf: add queue setup and release in split queue model
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8b7170f49e..270ab7b6de 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 02a55e0658..87a024b6db 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,10 +481,10 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, err, i;
+ int size, err, i, j;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -520,7 +520,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -550,7 +589,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -581,7 +620,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -612,7 +683,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int size, err, i;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -642,7 +713,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -700,7 +793,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 05/14] net/idpf: add support for queue start and stop
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 270ab7b6de..b625868ceb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 87a024b6db..da48eef29d 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -829,6 +829,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 06/14] net/idpf: add support for device information get
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 07/14] net/idpf: add support for packet type get
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index b625868ceb..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -210,6 +210,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -221,6 +222,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index da48eef29d..72c7d303a9 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1003,3 +1212,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 212 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 242 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..31e266bbec 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1748,14 +1911,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1948,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +1988,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2092,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 10/14] net/idpf: add support for RSS
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 31e266bbec..05976acf7f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 72c7d303a9..c5290d8050 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 11/14] net/idpf: add support for MTU configuration
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 12/14] net/idpf: add support for write back based on ITR expire
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index c5290d8050..c4aa744953 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1134,6 +1134,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 146 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1177 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 84b21e7be0..8f81894e27 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 05976acf7f..74ce84787a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2146,26 +2148,170 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v6 14/14] net/idpf: add support for timestamp offload
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-19 14:54 ` [PATCH v6 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-19 14:54 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-19 14:54 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 74ce84787a..db69dc9ce2 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 00/14] add support for idpf PMD in DPDK
2022-10-19 14:54 ` [PATCH v6 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU Ethernet ES2000 Series (Device ID: 0x1452).
v2:
fixed some coding style issues and did some refactors.
v3:
fixed some coding style issues and did some refactors.
v4:
fixed some coding style issues and build errors.
v5:
fixed typo.
v6-v7:
fixed build errors.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2375 +++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1445 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15363 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 01/14] common/idpf: introduce common library
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 375 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8509 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..20706bc2c6
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,375 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef enum idpf_status idpf_status;
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..183587b51a
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 02/14] net/idpf: add support for device initialization
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 5 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 204 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1617 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 2bd4a55f1b..e2d9c2ac50 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -758,6 +758,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..cc56db6e70
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+50/100/200 Gbps Intel® IPU Ethernet ES2000 Series Network Adapters.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 2da8bc9661..6697b61848 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -236,6 +236,11 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® IPU Ethernet ES2000 Series.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..77824d5f7f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 03/14] net/idpf: add queue setup and release in single queue model
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 251 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 994 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index cc56db6e70..84b21e7be0 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 77824d5f7f..8b7170f49e 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -197,6 +202,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..88cda54a26 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,256 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 04/14] net/idpf: add queue setup and release in split queue model
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8b7170f49e..270ab7b6de 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 88cda54a26..3578346bce 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,11 +481,11 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, i;
+ int size, i, j;
int err = 0;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -521,7 +521,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -551,7 +590,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -582,7 +621,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -614,7 +685,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int err = 0;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -644,7 +715,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -702,7 +795,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 05/14] net/idpf: add support for queue start and stop
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 270ab7b6de..b625868ceb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 3578346bce..6b6881872b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -831,6 +831,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 06/14] net/idpf: add support for device information get
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 07/14] net/idpf: add support for packet type get
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index b625868ceb..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -210,6 +210,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -221,6 +222,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 6b6881872b..7389128712 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1005,3 +1214,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 212 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 242 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..31e266bbec 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1748,14 +1911,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1948,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +1988,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2092,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 10/14] net/idpf: add support for RSS
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 31e266bbec..05976acf7f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 7389128712..8f447046b1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 11/14] net/idpf: add support for MTU configuration
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 12/14] net/idpf: add support for write back based on ITR expire
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 8f447046b1..1b5f021834 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1136,6 +1136,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 146 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1177 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 84b21e7be0..8f81894e27 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 05976acf7f..74ce84787a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2146,26 +2148,170 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v7 14/14] net/idpf: add support for timestamp offload
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-20 2:41 ` [PATCH v7 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-20 2:41 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 2:41 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 74ce84787a..db69dc9ce2 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 00/14] add support for idpf PMD in DPDK
2022-10-20 2:41 ` [PATCH v7 14/14] net/idpf: add support for timestamp offload Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v8:
fixed build errors.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 374 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 271 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2375 +++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1445 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15363 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 01/14] common/idpf: introduce common library
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 374 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8508 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..ee64884a97
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,374 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef __always_unused
+#define __always_unused __attribute__((__unused__))
+#endif
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((__unused__))
+#endif
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __attribute__((packed));
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __attribute__((packed));
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__attribute__((unused)) struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__attribute__((unused)) struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..183587b51a
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 02/14] net/idpf: add support for device initialization
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 204 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1618 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..428bf4266a
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..736fec6dcb 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -260,6 +260,12 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..77824d5f7f
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,204 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ int ret = rte_atomic32_cmpset(&adapter->pend_cmd, VIRTCHNL_OP_UNKNOWN, ops);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 03/14] net/idpf: add queue setup and release in single queue model
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 251 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 994 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 428bf4266a..fbc0f51de6 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 77824d5f7f..8b7170f49e 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -197,6 +202,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..88cda54a26 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,256 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 04/14] net/idpf: add queue setup and release in split queue model
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8b7170f49e..270ab7b6de 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 88cda54a26..3578346bce 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,11 +481,11 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, i;
+ int size, i, j;
int err = 0;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -521,7 +521,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -551,7 +590,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -582,7 +621,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -614,7 +685,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int err = 0;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -644,7 +715,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -702,7 +795,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 05/14] net/idpf: add support for queue start and stop
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 270ab7b6de..b625868ceb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -217,6 +217,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 3578346bce..6b6881872b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -831,6 +831,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 06/14] net/idpf: add support for device information get
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 07/14] net/idpf: add support for packet type get
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index b625868ceb..863d163330 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -210,6 +210,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -221,6 +222,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 6b6881872b..7389128712 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1005,3 +1214,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 212 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 242 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..31e266bbec 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1748,14 +1911,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1948,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +1988,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2092,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 10/14] net/idpf: add support for RSS
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 863d163330..bba8223cc4 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -214,6 +237,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 31e266bbec..05976acf7f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 7389128712..8f447046b1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 11/14] net/idpf: add support for MTU configuration
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 12/14] net/idpf: add support for write back based on ITR expire
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index bba8223cc4..505b3f36e3 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -248,6 +253,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 8f447046b1..1b5f021834 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1136,6 +1136,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 146 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1177 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index fbc0f51de6..cb21d5fd2f 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 505b3f36e3..9b0de772c9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 05976acf7f..74ce84787a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2146,26 +2148,170 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v8 14/14] net/idpf: add support for timestamp offload
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-20 6:29 ` [PATCH v8 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-20 6:29 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-20 6:29 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9b0de772c9..49919e5c74 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 74ce84787a..db69dc9ce2 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 00/14] add support for idpf PMD in DPDK
2022-10-20 6:29 ` [PATCH v8 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
` (13 more replies)
0 siblings, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
Junfeng Guo (14):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add queue setup and release in single queue model
net/idpf: add queue setup and release in split queue model
net/idpf: add support for queue start and stop
net/idpf: add support for device information get
net/idpf: add support for packet type get
net/idpf: add support for basic Rx/Tx datapath
net/idpf: add support for Rx/Tx offloading
net/idpf: add support for RSS
net/idpf: add support for MTU configuration
net/idpf: add support for write back based on ITR expire
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 24 +
doc/guides/nics/idpf.rst | 94 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 364 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1244 ++++++++
drivers/net/idpf/idpf_ethdev.h | 273 ++
drivers/net/idpf/idpf_logs.h | 42 +
drivers/net/idpf/idpf_rxtx.c | 2375 +++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 324 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1445 ++++++++++
drivers/net/idpf/meson.build | 45 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
37 files changed, 15355 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 01/14] common/idpf: introduce common library
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 6:40 ` Andrew Rybchenko
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 02/14] net/idpf: add support for device initialization Junfeng Guo
` (12 subsequent siblings)
13 siblings, 2 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data Path Function)
PMD.
Also add OS specific implementation about some MACRO definitions and
small functions which are specific for DPDK.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/idpf_alloc.h | 22 +
drivers/common/idpf/idpf_common.c | 364 +++
drivers/common/idpf/idpf_controlq.c | 691 +++++
drivers/common/idpf/idpf_controlq.h | 224 ++
drivers/common/idpf/idpf_controlq_api.h | 234 ++
drivers/common/idpf/idpf_controlq_setup.c | 179 ++
drivers/common/idpf/idpf_devids.h | 18 +
drivers/common/idpf/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/idpf_osdep.h | 364 +++
drivers/common/idpf/idpf_prototype.h | 45 +
drivers/common/idpf/idpf_type.h | 106 +
drivers/common/idpf/meson.build | 14 +
drivers/common/idpf/siov_regs.h | 47 +
drivers/common/idpf/version.map | 12 +
drivers/common/idpf/virtchnl.h | 2866 +++++++++++++++++++
drivers/common/idpf/virtchnl2.h | 1462 ++++++++++
drivers/common/idpf/virtchnl2_lan_desc.h | 606 ++++
drivers/common/idpf/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/meson.build | 1 +
21 files changed, 8498 insertions(+)
create mode 100644 drivers/common/idpf/idpf_alloc.h
create mode 100644 drivers/common/idpf/idpf_common.c
create mode 100644 drivers/common/idpf/idpf_controlq.c
create mode 100644 drivers/common/idpf/idpf_controlq.h
create mode 100644 drivers/common/idpf/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/idpf_devids.h
create mode 100644 drivers/common/idpf/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/idpf_osdep.h
create mode 100644 drivers/common/idpf/idpf_prototype.h
create mode 100644 drivers/common/idpf/idpf_type.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/siov_regs.h
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/common/idpf/virtchnl.h
create mode 100644 drivers/common/idpf/virtchnl2.h
create mode 100644 drivers/common/idpf/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/virtchnl_inline_ipsec.h
diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/idpf_common.c b/drivers/common/idpf/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/idpf_controlq.c b/drivers/common/idpf/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_controlq.h b/drivers/common/idpf/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_api.h b/drivers/common/idpf/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/idpf_controlq_setup.c b/drivers/common/idpf/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/idpf_devids.h b/drivers/common/idpf/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/idpf_lan_pf_regs.h b/drivers/common/idpf/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/idpf_lan_txrx.h b/drivers/common/idpf/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/idpf_lan_vf_regs.h b/drivers/common/idpf/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/idpf_osdep.h b/drivers/common/idpf/idpf_osdep.h
new file mode 100644
index 0000000000..f7ad63dff7
--- /dev/null
+++ b/drivers/common/idpf/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#ifndef BIT
+#define BIT(a) (1ULL << (a))
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte__packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/idpf_prototype.h b/drivers/common/idpf/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/idpf_type.h b/drivers/common/idpf/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..183587b51a
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
diff --git a/drivers/common/idpf/siov_regs.h b/drivers/common/idpf/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/idpf/virtchnl.h b/drivers/common/idpf/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/virtchnl2.h b/drivers/common/idpf/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/virtchnl2_lan_desc.h b/drivers/common/idpf/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/virtchnl_inline_ipsec.h b/drivers/common/idpf/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 02/14] net/idpf: add support for device initialization
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 7:39 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
` (11 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops skeleton:
- dev_configure
- dev_start
- dev_stop
- dev_close
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 15 +
doc/guides/nics/idpf.rst | 52 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 206 +++++++
drivers/net/idpf/idpf_logs.h | 42 ++
drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
drivers/net/idpf/meson.build | 16 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1620 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..f029a279b3
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Multiprocess aware = Y
+FreeBSD = Y
+Linux = Y
+Windows = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..428bf4266a
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,52 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+Windows Prerequisites
+---------------------
+
+- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
+ to setup the basic DPDK environment.
+
+- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
+
+- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
+
+- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
+ <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``not create ethdev``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..736fec6dcb 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -260,6 +260,12 @@ New Features
strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p'
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
Removed Items
-------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..7806c43668
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,774 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_VPORT "vport"
+
+struct idpf_adapter_list adapter_list;
+bool adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .dev_close = idpf_dev_close,
+ .link_update = idpf_dev_link_update,
+};
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (!adapter->vport_req_info[idx]) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (!adapter->vport_req_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+ vport->stopped = 1;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+{
+ return 0;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ vport->stopped = 0;
+
+ if (idpf_vc_ena_dis_vport(vport, true)) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ goto err_vport;
+ }
+
+ return 0;
+
+err_vport:
+ return -1;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (vport->stopped == 1)
+ return 0;
+
+ if (idpf_vc_ena_dis_vport(vport, false))
+ PMD_DRV_LOG(ERR, "disable vport failed");
+
+ vport->stopped = 1;
+ dev->data->dev_started = 0;
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return 0;
+
+ idpf_dev_stop(dev);
+ idpf_vc_destroy_vport(vport);
+
+ adapter->cur_vports &= ~BIT(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo))
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i))
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
+ adapter->cur_vports |= BIT(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (!devargs)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (!kvlist) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (!hw->asq || !hw->arq) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->mbx_resp) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (!adapter->caps) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter)) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (!adapter->vport_req_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (!adapter->vport_recv_info) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (!adapter->vports) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (!vports[i])
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ /* for secondary processes, we don't initialise any further as primary
+ * has already done this work.
+ */
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return ret;
+
+ dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &adapter_list, next) {
+ if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = FALSE;
+
+ if (!adapter_list_init) {
+ TAILQ_INIT(&adapter_list);
+ adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (!adapter) {
+ first_probe = TRUE;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (!adapter) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval)
+ PMD_DRV_LOG(ERR, "failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ TAILQ_REMOVE(&adapter_list, adapter, next);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..c2d72fae11
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
+#define IDPF_MAX_NUM_QUEUES 256
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_MAX_PKT_TYPE 1024
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint16_t num_tx_q;
+ uint16_t num_rx_q;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ /* Event from ipf */
+ bool link_up;
+ uint32_t link_speed;
+
+ uint16_t devarg_id;
+ bool stopped;
+ struct virtchnl2_vport_stats eth_stats_offset;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ uint16_t used_vecs_num;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+_clear_cmd(struct idpf_adapter *adapter)
+{
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline int
+_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ int ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct rte_eth_dev *dev);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..a64544f86e
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#define PMD_DRV_LOG_RAW(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
+ "%s(): " fmt "\n", __func__, ##args)
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, fmt, args...) \
+ RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ef0288ff45
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,495 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#include "idpf_prototype.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg)
+ break;
+ }
+ if (err)
+ goto error;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+error:
+ return err;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err = 0;
+
+ err = idpf_vc_clean(adapter);
+ if (err)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (!ctlq_msg) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (!dma_mem) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (!dma_mem->va) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err)
+ goto send_error;
+
+ return err;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret && dma_mem)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int ret, err = 0, i = 0;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (_atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ _clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ case VIRTCHNL2_OP_GET_STATS:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ _clear_cmd(adapter);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ _clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return err;
+}
+
+int
+idpf_vc_create_vport(struct rte_eth_dev *dev)
+{
+ struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (!adapter->vport_recv_info[idx]) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!adapter->vport_recv_info[idx]) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return err;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return err;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+ return err;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..bf8bf58ef5
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+includes += include_directories('../../common/idpf')
+deps += ['common_idpf', 'security', 'cryptodev']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 7:44 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
` (10 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in single queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
doc/guides/nics/idpf.rst | 22 ++
drivers/net/idpf/idpf_ethdev.c | 58 ++++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
drivers/net/idpf/idpf_vchnl.c | 251 ++++++++++++++++
drivers/net/idpf/meson.build | 1 +
8 files changed, 994 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index f029a279b3..681a908194 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,8 @@
; is selected.
;
[Features]
+Runtime Rx queue setup = Y
+Runtime Tx queue setup = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index 428bf4266a..fbc0f51de6 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -45,6 +45,28 @@ Runtime Config Options
Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
Driver compilation and testing
------------------------------
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 7806c43668..96af54f47b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,13 +10,18 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
#define IDPF_VPORT "vport"
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
IDPF_VPORT,
NULL
};
@@ -52,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.link_update = idpf_dev_link_update,
};
@@ -81,6 +90,18 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
return 0;
}
@@ -110,6 +131,8 @@ idpf_init_vport(struct rte_eth_dev *dev)
int i;
vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
vport->num_rx_q = vport_info->num_rx_q;
vport->max_mtu = vport_info->max_mtu;
@@ -149,6 +172,12 @@ idpf_init_vport(struct rte_eth_dev *dev)
static int
idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
{
+ if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
+ dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
+ PMD_INIT_LOG(ERR, "Invalid queue number.");
+ return -EINVAL;
+ }
+
return 0;
}
@@ -312,6 +341,25 @@ parse_vport(const char *key, const char *value, void *args)
return 0;
}
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
static int
idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
{
@@ -333,6 +381,16 @@ idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
if (ret)
goto bail;
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret)
+ goto bail;
+
bail:
rte_kvargs_free(kvlist);
return ret;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c2d72fae11..1d1769d199 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -83,6 +83,8 @@ struct idpf_chunks_info {
struct idpf_vport {
struct idpf_adapter *adapter; /* Backreference to associated adapter */
uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
uint16_t num_tx_q;
uint16_t num_rx_q;
@@ -118,6 +120,9 @@ struct idpf_adapter {
uint32_t cmd_retval; /* return value of the cmd response from ipf */
uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+ uint32_t txq_model;
+ uint32_t rxq_model;
+
/* Vport info */
uint8_t **vport_req_info;
uint8_t **vport_recv_info;
@@ -199,6 +204,10 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..bff90dd9c6
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,465 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (!rxq->sw_ring)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (!txq || !txq->sw_ring) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc) {
+ nb_desc = 0;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (!q)
+ return;
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (!q)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return -1;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return -1;
+}
+
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..69a1fa6348
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_osdep.h"
+#include "idpf_type.h"
+#include "idpf_devids.h"
+#include "idpf_lan_txrx.h"
+#include "idpf_lan_pf_regs.h"
+#include "virtchnl.h"
+#include "virtchnl2.h"
+#include "virtchnl2_lan_desc.h"
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+/* used for Rx Bulk Allocate */
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_TX_MAX_BURST 32
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
+ /* for rx bulk */
+ uint16_t rx_nb_avail; /* number of staged packets ready */
+ uint16_t rx_next_avail; /* index of next staged packets */
+ uint16_t rx_free_trigger; /* triggers rx buffer allocation */
+ struct rte_mbuf *rx_stage[IDPF_RX_MAX_BURST * 2]; /* store mbuf */
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+ uint8_t vlan_flag;
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ef0288ff45..88cda54a26 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#include "idpf_prototype.h"
@@ -469,6 +470,256 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ return -1;
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ return -1;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index bf8bf58ef5..832f887296 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -12,5 +12,6 @@ deps += ['common_idpf', 'security', 'cryptodev']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 04/14] net/idpf: add queue setup and release in split queue model
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 05/14] net/idpf: add support for queue start and stop Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations in split queue model:
- rx_queue_setup
- rx_queue_release
- tx_queue_setup
- tx_queue_release
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 34 ++-
drivers/net/idpf/idpf_ethdev.h | 11 +
drivers/net/idpf/idpf_rxtx.c | 483 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_vchnl.c | 127 ++++++++-
4 files changed, 642 insertions(+), 13 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 96af54f47b..f9717cd5ff 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,13 +90,25 @@ idpf_init_vport_req_info(struct rte_eth_dev *dev)
(struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
- if (adapter->txq_model) {
+ if (!adapter->txq_model) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
vport_info->txq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
vport_info->num_tx_complq = 0;
}
- if (adapter->rxq_model) {
+ if (!adapter->rxq_model) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
vport_info->rxq_model =
rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
@@ -134,7 +146,9 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->txq_model = vport_info->txq_model;
vport->rxq_model = vport_info->rxq_model;
vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
@@ -157,6 +171,22 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport_info->chunks.chunks[i].qtail_reg_start;
vport->chunks_info.rx_qtail_spacing =
vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
}
}
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 1d1769d199..49d71154aa 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -73,11 +73,18 @@ enum idpf_vc_result {
struct idpf_chunks_info {
uint32_t tx_start_qid;
uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
uint64_t tx_qtail_start;
uint32_t tx_qtail_spacing;
uint64_t rx_qtail_start;
uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
};
struct idpf_vport {
@@ -86,7 +93,11 @@ struct idpf_vport {
uint32_t txq_model;
uint32_t rxq_model;
uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bff90dd9c6..82cc1d8f05 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -98,7 +98,8 @@ release_txq_mbufs(struct idpf_tx_queue *txq)
}
if (txq->sw_nb_desc) {
- nb_desc = 0;
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
} else {
/* For single queue model */
nb_desc = txq->nb_tx_desc;
@@ -127,6 +128,21 @@ idpf_rx_queue_release(void *rxq)
if (!q)
return;
+ /* Split queue */
+ if (q->bufq1 && q->bufq2) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
/* Single queue */
q->ops->release_mbufs(q);
rte_free(q->sw_ring);
@@ -149,6 +165,65 @@ idpf_tx_queue_release(void *txq)
rte_free(q);
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (!rxq)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static inline void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -181,6 +256,58 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->rxrearm_nb = 0;
}
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (!txq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (!cq) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
static inline void
reset_single_tx_queue(struct idpf_tx_queue *txq)
{
@@ -218,6 +345,224 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx]) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!rxq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq1) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!bufq2) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
static int
idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
@@ -337,7 +682,138 @@ idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, rx_conf, mp);
else
- return -1;
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ PMD_INIT_FUNC_TRACE();
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh))
+ return -EINVAL;
+
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx]) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (!txq->sw_ring) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (!cq) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (!mz) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
}
static int
@@ -449,7 +925,8 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
else
- return -1;
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
}
void
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 88cda54a26..3578346bce 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -481,11 +481,11 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t total_qs, num_qs;
- int size, i;
+ int size, i, j;
int err = 0;
int k = 0;
- total_qs = vport->num_rx_q;
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
while (total_qs) {
if (total_qs > adapter->max_rxq_per_msg) {
num_qs = adapter->max_rxq_per_msg;
@@ -521,7 +521,46 @@ idpf_vc_config_rxqs(struct idpf_vport *vport)
rxq_info->ring_len = rxq[k]->nb_rx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
}
memset(&args, 0, sizeof(args));
args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
@@ -551,7 +590,7 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
struct virtchnl2_rxq_info *rxq_info;
struct idpf_cmd_info args;
uint16_t num_qs;
- int size, err;
+ int size, err, i;
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
num_qs = IDPF_RXQ_PER_GRP;
@@ -582,7 +621,39 @@ idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
} else {
- return -1;
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
}
memset(&args, 0, sizeof(args));
@@ -614,7 +685,7 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
int err = 0;
int k = 0;
- total_qs = vport->num_tx_q;
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
while (total_qs) {
if (total_qs > adapter->max_txq_per_msg) {
num_qs = adapter->max_txq_per_msg;
@@ -644,7 +715,29 @@ idpf_vc_config_txqs(struct idpf_vport *vport)
txq_info->ring_len = txq[k]->nb_tx_desc;
}
} else {
- return -1;
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
}
memset(&args, 0, sizeof(args));
@@ -702,7 +795,25 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
txq_info->ring_len = txq[txq_id]->nb_tx_desc;
} else {
- return -1;
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
}
memset(&args, 0, sizeof(args));
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 05/14] net/idpf: add support for queue start and stop
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 7:53 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 06/14] net/idpf: add support for device information get Junfeng Guo
` (8 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_start
- rx_queue_stop
- tx_queue_start
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 46 +++++
drivers/net/idpf/idpf_ethdev.h | 3 +
drivers/net/idpf/idpf_rxtx.c | 309 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 150 +++++++++++++++
6 files changed, 515 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 681a908194..597beec5d9 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,6 +7,7 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
Multiprocess aware = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9717cd5ff..c25f222c5e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,10 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
@@ -211,6 +215,39 @@ idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -220,6 +257,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (idpf_start_queues(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ goto err_mtu;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -228,6 +270,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
return 0;
err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
return -1;
}
@@ -244,6 +288,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
if (idpf_vc_ena_dis_vport(vport, false))
PMD_DRV_LOG(ERR, "disable vport failed");
+ idpf_stop_queues(dev);
+
vport->stopped = 1;
dev->data->dev_started = 0;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 49d71154aa..9608204e4c 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -219,6 +219,9 @@ int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 82cc1d8f05..95193713c4 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -929,6 +929,289 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!mbuf)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (!rxq || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (!rxq->bufq1) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ PMD_DRV_FUNC_TRACE();
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ PMD_DRV_FUNC_TRACE();
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
void
idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
@@ -940,3 +1223,29 @@ idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
{
idpf_tx_queue_release(dev->data->tx_queues[qid]);
}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (!rxq)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i))
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 69a1fa6348..f0427b96c5 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -175,12 +175,18 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void idpf_stop_queues(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 3578346bce..6b6881872b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -831,6 +831,156 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (!queue_select)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 06/14] net/idpf: add support for device information get
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 7:56 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 07/14] net/idpf: add support for packet type get Junfeng Guo
` (7 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops dev_infos_get.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 50 ++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c25f222c5e..d1b6797d4a 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,6 +30,8 @@ static int idpf_dev_configure(struct rte_eth_dev *dev);
static int idpf_dev_start(struct rte_eth_dev *dev);
static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -65,9 +67,57 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+ dev_info->default_rxportconf.burst_size = IDPF_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = IDPF_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 07/14] net/idpf: add support for packet type get
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 8:00 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 2 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 235 ++++++++++++++++++++++++++++++
6 files changed, 270 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 597beec5d9..79ffafc0d4 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d1b6797d4a..20f0f39640 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -55,6 +55,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -652,6 +653,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (!adapter->caps) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 9608204e4c..d414efc917 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -212,6 +212,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -223,6 +224,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 95193713c4..e92ea09c3b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index f0427b96c5..cd1dda4688 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -47,6 +47,10 @@
#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
(RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -189,4 +193,6 @@ void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 6b6881872b..7389128712 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -303,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ _clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -1005,3 +1214,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (!ptype_info)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 08/14] net/idpf: add support for basic Rx/Tx datapath
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx & Tx support in split queue mode and single queue mode.
Split queue mode is selected by default.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 8 +-
drivers/net/idpf/idpf_rxtx.c | 667 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 16 +-
3 files changed, 687 insertions(+), 4 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 20f0f39640..c6d6c87274 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -313,6 +313,9 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true)) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
@@ -764,8 +767,11 @@ idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
/* for secondary processes, we don't initialise any further as primary
* has already done this work.
*/
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
return ret;
+ }
dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index e92ea09c3b..6280b7887b 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1268,3 +1268,670 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta))) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(!rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill))) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (!pkt_len)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (!bufq_id) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely(!(txq->last_desc_cleaned % 32))) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely(!(tx_id % 32)))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(!rxq) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+ rxq->hw_register_set = 1;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if (!(rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(!nmb)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(!txq) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq)) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if (!(ol_flags & RTE_MBUF_F_TX_TCP_SEG)) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index cd1dda4688..bd3ebe2f50 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,9 +44,6 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
-#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
- (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
-
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -191,8 +188,21 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 8:29 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx/Tx offloading support, including TSO and CHKSUM.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 3 +
drivers/net/idpf/idpf_ethdev.c | 10 ++
drivers/net/idpf/idpf_rxtx.c | 212 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 19 +++
4 files changed, 242 insertions(+), 2 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 79ffafc0d4..36dec39b9f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,9 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+TSO = P
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c6d6c87274..ffde2ce7d1 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6280b7887b..31e266bbec 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if (flags & RTE_MBUF_F_TX_TCP_SEG)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (!tx_offload.l4_len) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(!txq) || unlikely(!txq->q_started))
return nb_tx;
@@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (unlikely(!(tx_id % 32)))
txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
}
@@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1748,14 +1911,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1782,11 +1948,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1814,6 +1988,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
m_seg = tx_pkt;
do {
@@ -1896,11 +2092,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+ if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
+
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index bd3ebe2f50..a2394ffa2c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -44,6 +44,25 @@
#define IDPF_MAX_TSO_FRAME_SIZE 262143
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG)
+
+#define IDPF_TX_OFFLOAD_MASK ( \
+ RTE_MBUF_F_TX_OUTER_IPV6 | \
+ RTE_MBUF_F_TX_OUTER_IPV4 | \
+ RTE_MBUF_F_TX_IPV6 | \
+ RTE_MBUF_F_TX_IPV4 | \
+ RTE_MBUF_F_TX_VLAN | \
+ RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_ETH_TX_OFFLOAD_SECURITY)
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 10/14] net/idpf: add support for RSS
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 8:38 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 11/14] net/idpf: add support for MTU configuration Junfeng Guo
` (3 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
4 files changed, 269 insertions(+), 3 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ffde2ce7d1..cc1cfe402b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
@@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#ifndef IDPF_RSS_KEY_LEN
+#define IDPF_RSS_KEY_LEN 52
+#endif
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
}
static int
-idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (!vport->rss_key) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (!vport->rss_lut) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_lut;
+ }
+
+ if (!rss_conf->rss_key) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ RTE_MIN(rss_conf->rss_key_len,
+ vport->rss_key_size));
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg;
+ }
+
+ return ret;
+
+err_cfg:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_key:
+ return ret;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
PMD_INIT_LOG(ERR, "Invalid queue number.");
return -EINVAL;
}
- return 0;
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |=
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ if (adapter->caps->rss_caps) {
+ ret = idpf_init_rss(vport);
+ if (ret) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ }
+
+ return ret;
}
static int
@@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_dev_stop(dev);
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index d414efc917..c25fa2e3ce 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -56,6 +56,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -102,11 +116,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -216,6 +239,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct rte_eth_dev *dev);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 31e266bbec..05976acf7f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
return flags;
}
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
rxm->ol_flags |= pkt_flags;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 7389128712..8f447046b1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (!rss_key)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+ return err;
+ }
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (!rss_lut)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 11/14] net/idpf: add support for MTU configuration
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 24 ++++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 36dec39b9f..ee3f6e7c6f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -10,6 +10,7 @@
Queue start/stop = Y
Runtime Rx queue setup = Y
Runtime Tx queue setup = Y
+MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cc1cfe402b..803c0d7e12 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -32,6 +32,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -70,6 +71,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -83,6 +85,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
@@ -131,6 +136,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -429,6 +446,13 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->stopped = 0;
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ goto err_mtu;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 12/14] net/idpf: add support for write back based on ITR expire
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 11/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Force write-backs by setting WB_ON_ITR bit in DYN_CTL register,
so that the packets can be received once at a time.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 119 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 8 +++
drivers/net/idpf/idpf_vchnl.c | 107 +++++++++++++++++++++++++++++
3 files changed, 234 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 803c0d7e12..ac70c029b0 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -404,6 +404,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return ret;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (!qv_map) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true)) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -441,6 +525,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
PMD_INIT_FUNC_TRACE();
@@ -453,6 +541,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev)) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev)) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -489,6 +594,12 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ if (idpf_vc_config_irq_map_unmap(vport, false))
+ PMD_DRV_LOG(ERR, "config interrupt unmapping failed");
+
+ if (idpf_vc_dealloc_vectors(vport))
+ PMD_DRV_LOG(ERR, "deallocate interrupt vectors failed");
+
vport->stopped = 1;
dev->data->dev_started = 0;
@@ -513,6 +624,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
rte_free(vport->rss_key);
vport->rss_key = NULL;
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
adapter->cur_vports &= ~BIT(vport->devarg_id);
rte_free(vport);
@@ -864,6 +981,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c25fa2e3ce..1792292bbb 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -130,6 +130,11 @@ struct idpf_vport {
uint8_t *rss_key;
uint64_t rss_hf;
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -250,6 +255,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 8f447046b1..1b5f021834 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -1136,6 +1136,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (!map_info)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (!alloc_vec)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (!vport->recv_vectors) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (!vport->recv_vectors) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 20 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 146 ++++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1177 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index fbc0f51de6..cb21d5fd2f 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -72,3 +72,23 @@ Driver compilation and testing
Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
for details.
+
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 1792292bbb..80171f1251 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -182,6 +182,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 05976acf7f..74ce84787a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -2146,26 +2148,170 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (!rxq->sw_ring || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i])
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+ }
+#endif /* RTE_ARCH_X86 */
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (!txq)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a2394ffa2c..8f440557bd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -211,12 +211,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..e25f277405
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(!cache))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if (!(rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)))
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (!cache || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (!txq->sw_ring || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..aa66614b0d
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (!rxq)
+ return -1;
+
+ if (!rte_is_power_of_2(rxq->nb_rx_desc))
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if (rxq->nb_rx_desc % rxq->rx_free_thresh)
+ return -1;
+
+ if (rxq->offloads & ICE_RX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (!txq)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if (txq->offloads & ICE_TX_NO_VECTOR_FLAGS)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index 832f887296..c96e4bfcaa 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -15,3 +15,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v9 14/14] net/idpf: add support for timestamp offload
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-21 5:18 ` [PATCH v9 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-21 5:18 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-21 5:18 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 57 ++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index ee3f6e7c6f..145d995f51 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -14,6 +14,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Multiprocess aware = Y
FreeBSD = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ac70c029b0..c3f628655c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -19,6 +19,8 @@
struct idpf_adapter_list adapter_list;
bool adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -97,7 +99,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 80171f1251..a2f8d8e45d 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -187,6 +187,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 74ce84787a..db69dc9ce2 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -950,6 +952,23 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
socket_id, tx_conf);
}
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1043,6 +1062,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (!rxq->bufq1) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1413,6 +1439,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1422,9 +1449,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1494,6 +1523,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1797,18 +1838,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(!rxq) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1878,6 +1923,18 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 8f440557bd..04047f5e56 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -16,6 +16,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -67,6 +102,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -231,4 +268,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 01/14] common/idpf: introduce common library
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
@ 2022-10-21 6:40 ` Andrew Rybchenko
2022-10-21 12:35 ` Xing, Beilei
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
1 sibling, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 6:40 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiao Wang
On 10/21/22 08:18, Junfeng Guo wrote:
> Introduce common library for IDPF (Infrastructure Data Path Function)
> PMD.
>
> Also add OS specific implementation about some MACRO definitions and
> small functions which are specific for DPDK.
Common drivers are required when different class drivers need
to share some code. So, it must be expalined here why do you
create common driver instead of usage of base/ driver in your
net driver.
Note that common driver is a DPDK driver and it must follow
DPDK coding style. If the code is actually shared with
something else and do not follow DPDK coding style from the
very beginning (since it is an existing code), it should be
in base/ subdir in either common driver or net driver.
Also you should not use own trivial wrappers for DPDK API
in DPDK-specific code. It just complicates reading.
E.g. BIT() vs RTE_BIT32().
So, I need an answer on above questions before I continue
review.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> diff --git a/drivers/common/idpf/idpf_alloc.h b/drivers/common/idpf/idpf_alloc.h
> new file mode 100644
> index 0000000000..bc054851b3
> --- /dev/null
> +++ b/drivers/common/idpf/idpf_alloc.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2001-2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_ALLOC_H_
> +#define _IDPF_ALLOC_H_
> +
> +/* Memory types */
If it is a DPDK-specific driver and it is an interface provided
by common driver, it should use Doxygen-style comments to be a
part of genereated API documentation.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 02/14] net/idpf: add support for device initialization
2022-10-21 5:18 ` [PATCH v9 02/14] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-21 7:39 ` Andrew Rybchenko
2022-10-21 7:48 ` Andrew Rybchenko
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 7:39 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang
On 10/21/22 08:18, Junfeng Guo wrote:
> Support device init and add the following dev ops skeleton:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> MAINTAINERS | 9 +
> doc/guides/nics/features/idpf.ini | 15 +
> doc/guides/nics/idpf.rst | 52 ++
> doc/guides/nics/index.rst | 1 +
> doc/guides/rel_notes/release_22_11.rst | 6 +
> drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
> drivers/net/idpf/idpf_ethdev.h | 206 +++++++
> drivers/net/idpf/idpf_logs.h | 42 ++
> drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
> drivers/net/idpf/meson.build | 16 +
> drivers/net/idpf/version.map | 3 +
> drivers/net/meson.build | 1 +
> 12 files changed, 1620 insertions(+)
> create mode 100644 doc/guides/nics/features/idpf.ini
> create mode 100644 doc/guides/nics/idpf.rst
> create mode 100644 drivers/net/idpf/idpf_ethdev.c
> create mode 100644 drivers/net/idpf/idpf_ethdev.h
> create mode 100644 drivers/net/idpf/idpf_logs.h
> create mode 100644 drivers/net/idpf/idpf_vchnl.c
> create mode 100644 drivers/net/idpf/meson.build
> create mode 100644 drivers/net/idpf/version.map
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 92b381bc30..34f8b9cc61 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -764,6 +764,15 @@ F: drivers/net/ice/
> F: doc/guides/nics/ice.rst
> F: doc/guides/nics/features/ice.ini
>
> +Intel idpf
> +M: Jingjing Wu <jingjing.wu@intel.com>
> +M: Beilei Xing <beilei.xing@intel.com>
> +T: git://dpdk.org/next/dpdk-next-net-intel
> +F: drivers/net/idpf/
> +F: drivers/common/idpf/
> +F: doc/guides/nics/idpf.rst
> +F: doc/guides/nics/features/idpf.ini
> +
> Intel igc
> M: Junfeng Guo <junfeng.guo@intel.com>
> M: Simei Su <simei.su@intel.com>
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> new file mode 100644
> index 0000000000..f029a279b3
> --- /dev/null
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -0,0 +1,15 @@
> +;
> +; Supported features of the 'idpf' network poll mode driver.
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +; A feature with "P" indicates only be supported when non-vector path
> +; is selected.
> +;
> +[Features]
> +Multiprocess aware = Y
Is multi-process support tested?
> +FreeBSD = Y
Have you actually checked it on FreeBSD?
> +Linux = Y
> +Windows = Y
It looks ridiculous taking into account that meson.build says
that "not supported on Windows".
> +x86-32 = Y
> +x86-64 = Y
> diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
> new file mode 100644
> index 0000000000..428bf4266a
> --- /dev/null
> +++ b/doc/guides/nics/idpf.rst
> @@ -0,0 +1,52 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> + Copyright(c) 2022 Intel Corporation.
> +
> +IDPF Poll Mode Driver
> +======================
> +
> +The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
> +Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
> +
Coding style requires two empty lines before the next section.
Here and everywhere below.
> +Linux Prerequisites
> +-------------------
> +
> +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
> +
> +- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
> + section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
> +
> +Windows Prerequisites
> +---------------------
Sorry, it does not make sense since build on Windows is not
supported.
> +
> +- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
> + to setup the basic DPDK environment.
> +
> +- Identify the Intel® Ethernet adapter and get the latest NVM/FW version.
> +
> +- To access any Intel® Ethernet hardware, load the NetUIO driver in place of existing built-in (inbox) driver.
> +
> +- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods repository
> + <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
> +
> +Pre-Installation Configuration
> +------------------------------
> +
> +Runtime Config Options
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +- ``vport`` (default ``not create ethdev``)
> +
> + The IDPF PMD supports creation of multiple vports for one PCI device, each vport
> + corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
> + can specify the vports with specific ID to be created, for example::
> +
> + -a ca:00.0,vport=[0,2,3]
> +
> + Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
> + NOTE: This parameter is MUST, otherwise there'll be no any ethdev created.
It hardly user-friendly. Of course, you know better, but I'd
consider to have some sensible default which is not NOP.
> +
> +Driver compilation and testing
> +------------------------------
> +
> +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
> +for details.
[snip]
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> new file mode 100644
> index 0000000000..7806c43668
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -0,0 +1,774 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#define IDPF_VPORT "vport"
> +
> +struct idpf_adapter_list adapter_list;
> +bool adapter_list_init;
If it is global, it should be exmplained what it is and why
locking is not required.
> +
> +static const char * const idpf_valid_args[] = {
> + IDPF_VPORT,
> + NULL
> +};
> +
> +static int idpf_dev_configure(struct rte_eth_dev *dev);
> +static int idpf_dev_start(struct rte_eth_dev *dev);
> +static int idpf_dev_stop(struct rte_eth_dev *dev);
> +static int idpf_dev_close(struct rte_eth_dev *dev);
> +static void idpf_adapter_rel(struct idpf_adapter *adapter);
> +
> +int
> +idpf_dev_link_update(struct rte_eth_dev *dev,
> + __rte_unused int wait_to_complete)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct rte_eth_link new_link;
> +
> + memset(&new_link, 0, sizeof(new_link));
> +
> + new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
> +
> + new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
> + new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
> + RTE_ETH_LINK_DOWN;
> + new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
> + RTE_ETH_LINK_SPEED_FIXED);
> +
> + return rte_eth_linkstatus_set(dev, &new_link);
> +}
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_start = idpf_dev_start,
> + .dev_stop = idpf_dev_stop,
> + .dev_close = idpf_dev_close,
> + .link_update = idpf_dev_link_update,
> +};
> +
> +static int
> +idpf_init_vport_req_info(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_create_vport *vport_info;
> + uint16_t idx = adapter->cur_vport_idx;
> +
> + if (idx == IDPF_INVALID_VPORT_IDX) {
> + PMD_INIT_LOG(ERR, "Invalid vport index.");
> + return -1;
> + }
> +
> + if (!adapter->vport_req_info[idx]) {
Compare vs NULL as DPDK coding style says.
> + adapter->vport_req_info[idx] = rte_zmalloc(NULL,
> + sizeof(struct virtchnl2_create_vport), 0);
> + if (!adapter->vport_req_info[idx]) {
Compare vs NULL. Please, review the code yourself
since otherwise we can play ping-pong for a long-long
time.
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
> + return -1;
> + }
> + }
> +
> + vport_info =
> + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
> +
> + vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
> +
> + return 0;
> +}
> +
> +static uint16_t
> +idpf_parse_devarg_id(char *name)
> +{
> + uint16_t val;
> + char *p;
> +
> + p = strstr(name, "vport_");
> + p += sizeof("vport_") - 1;
> +
> + val = strtoul(p, NULL, 10);
> +
> + return val;
> +}
> +
> +static int
> +idpf_init_vport(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t idx = adapter->cur_vport_idx;
> + struct virtchnl2_create_vport *vport_info =
> + (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
> + int i;
> +
> + vport->vport_id = vport_info->vport_id;
> + vport->num_tx_q = vport_info->num_tx_q;
> + vport->num_rx_q = vport_info->num_rx_q;
> + vport->max_mtu = vport_info->max_mtu;
> + rte_memcpy(vport->default_mac_addr,
> + vport_info->default_mac_addr, ETH_ALEN);
> + vport->sw_idx = idx;
> +
> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> + if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX) {
> + vport->chunks_info.tx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX) {
> + vport->chunks_info.rx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + }
> + }
> +
> + vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
> + vport->dev_data = dev->data;
> + vport->stopped = 1;
> +
> + adapter->vports[idx] = vport;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
> +{
Sorry, but you need to check configuration here and reject
unsupported configuration options which are not rejected by
ethdev layer.
> + return 0;
> +}
> +
> +static int
> +idpf_dev_start(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
Can we call it in a secondary process? Will it work?
> +
> + vport->stopped = 0;
> +
> + if (idpf_vc_ena_dis_vport(vport, true)) {
Compare vs 0 since return value is not bool.
Please, review all conditions yourself.
> + PMD_DRV_LOG(ERR, "Failed to enable vport");
> + goto err_vport;
> + }
> +
> + return 0;
> +
> +err_vport:
> + return -1;
> +}
> +
> +static int
> +idpf_dev_stop(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
What about secondary process? Will it work?
> +
> + if (vport->stopped == 1)
In therory, rte_eth_dev_stop() cares about it. If we need to
care about it here as well, it should be explaiend in a comment
why.
> + return 0;
> +
> + if (idpf_vc_ena_dis_vport(vport, false))
> + PMD_DRV_LOG(ERR, "disable vport failed");
Don't you want to propagate the failure further
(since operation can return status)?
Otherwise, it should be clear in logs that the failure is
ignored.
> +
> + vport->stopped = 1;
> + dev->data->dev_started = 0;
rte_eth_dev_stop() does it. So, it is not necessary here.
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_close(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return 0;
> +
> + idpf_dev_stop(dev);
> + idpf_vc_destroy_vport(vport);
> +
> + adapter->cur_vports &= ~BIT(vport->devarg_id);
Why not RTE_BIT32()?
> +
> + rte_free(vport);
> + dev->data->dev_private = NULL;
> +
> + return 0;
> +}
> +
> +static int
> +insert_value(struct idpf_adapter *adapter, uint16_t id)
> +{
> + uint16_t i;
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if (adapter->req_vports[i] == id)
> + return 0;
> + }
> +
> + if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + adapter->req_vports[adapter->req_vport_nb] = id;
> + adapter->req_vport_nb++;
> +
> + return 0;
> +}
> +
> +static const char *
> +parse_range(const char *value, struct idpf_adapter *adapter)
> +{
> + uint16_t lo, hi, i;
> + int n = 0;
> + int result;
> + const char *pos = value;
> +
> + result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
> + if (result == 1) {
> + if (lo >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + if (insert_value(adapter, lo))
> + return NULL;
> + } else if (result == 2) {
> + if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + for (i = lo; i <= hi; i++) {
> + if (insert_value(adapter, i))
> + return NULL;
> + }
> + } else {
> + return NULL;
> + }
> +
> + return pos + n;
> +}
> +
> +static int
> +parse_vport(const char *key, const char *value, void *args)
> +{
> + struct idpf_adapter *adapter = (struct idpf_adapter *)args;
> + const char *pos = value;
> + int i;
> +
> + adapter->req_vport_nb = 0;
> +
> + if (*pos == '[')
> + pos++;
> +
> + while (1) {
> + pos = parse_range(pos, adapter);
> + if (pos == NULL) {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> + if (*pos != ',')
> + break;
> + pos++;
> + }
> +
> + if (*value == '[' && *pos != ']') {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> +
> + if (adapter->cur_vport_nb + adapter->req_vport_nb >
> + IDPF_MAX_VPORT_NUM) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
> + adapter->cur_vports |= BIT(adapter->req_vports[i]);
> + adapter->cur_vport_nb++;
> + } else {
> + PMD_INIT_LOG(ERR, "Vport %d has been created",
> + adapter->req_vports[i]);
> + return -1;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int
> +idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
> +{
> + struct rte_devargs *devargs = pci_dev->device.devargs;
> + struct rte_kvargs *kvlist;
> + int ret;
> +
> + if (!devargs)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
> + if (!kvlist) {
> + PMD_INIT_LOG(ERR, "invalid kvargs key");
> + return -EINVAL;
> + }
> +
> + ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
> + adapter);
> + if (ret)
> + goto bail;
> +
> +bail:
> + rte_kvargs_free(kvlist);
> + return ret;
> +}
> +
> +static void
> +idpf_reset_pf(struct idpf_hw *hw)
> +{
> + uint32_t reg;
> +
> + reg = IDPF_READ_REG(hw, PFGEN_CTRL);
> + IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
> +}
> +
> +#define IDPF_RESET_WAIT_CNT 100
> +static int
> +idpf_check_pf_reset_done(struct idpf_hw *hw)
> +{
> + uint32_t reg;
> + int i;
> +
> + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
> + reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
> + if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
> + return 0;
> + rte_delay_ms(1000);
> + }
> +
> + PMD_INIT_LOG(ERR, "IDPF reset timeout");
> + return -EBUSY;
> +}
> +
> +#define CTLQ_NUM 2
> +static int
> +idpf_init_mbx(struct idpf_hw *hw)
> +{
> + struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
> + {
> + .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ATQH,
> + .tail = PF_FW_ATQT,
> + .len = PF_FW_ATQLEN,
> + .bah = PF_FW_ATQBAH,
> + .bal = PF_FW_ATQBAL,
> + .len_mask = PF_FW_ATQLEN_ATQLEN_M,
> + .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
> + .head_mask = PF_FW_ATQH_ATQH_M,
> + }
> + },
> + {
> + .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ARQH,
> + .tail = PF_FW_ARQT,
> + .len = PF_FW_ARQLEN,
> + .bah = PF_FW_ARQBAH,
> + .bal = PF_FW_ARQBAL,
> + .len_mask = PF_FW_ARQLEN_ARQLEN_M,
> + .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
> + .head_mask = PF_FW_ARQH_ARQH_M,
> + }
> + }
> + };
> + struct idpf_ctlq_info *ctlq;
> + int ret;
> +
> + ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
> + if (ret)
> + return ret;
> +
> + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
> + struct idpf_ctlq_info, cq_list) {
> + if (ctlq->q_id == IDPF_CTLQ_ID &&
> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
> + hw->asq = ctlq;
> + if (ctlq->q_id == IDPF_CTLQ_ID &&
> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
> + hw->arq = ctlq;
> + }
> +
> + if (!hw->asq || !hw->arq) {
> + idpf_ctlq_deinit(hw);
> + ret = -ENOENT;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
> +{
> + struct idpf_hw *hw = &adapter->hw;
> + int ret = 0;
> +
> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
> + hw->back = adapter;
> + hw->vendor_id = pci_dev->id.vendor_id;
> + hw->device_id = pci_dev->id.device_id;
> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
> +
> + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
> +
> + idpf_reset_pf(hw);
> + ret = idpf_check_pf_reset_done(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
> + goto err;
> + }
> +
> + ret = idpf_init_mbx(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
> + goto err;
> + }
> +
> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
> + IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->mbx_resp) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
> + goto err_mbx;
> + }
> +
> + if (idpf_vc_check_api_version(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to check api version");
> + goto err_api;
> + }
> +
> + adapter->caps = rte_zmalloc("idpf_caps",
> + sizeof(struct virtchnl2_get_capabilities), 0);
> + if (!adapter->caps) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
> + goto err_api;
> + }
> +
> + if (idpf_vc_get_caps(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
> + goto err_caps;
> + }
> +
> + adapter->max_vport_nb = adapter->caps->max_vports;
> +
> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_req_info),
> + 0);
> + if (!adapter->vport_req_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
> + goto err_caps;
> + }
> +
> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_recv_info),
> + 0);
> + if (!adapter->vport_recv_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
> + goto err_vport_recv_info;
> + }
> +
> + adapter->vports = rte_zmalloc("vports",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vports),
> + 0);
> + if (!adapter->vports) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
> + goto err_vports;
> + }
> +
> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_rx_queues)) /
> + sizeof(struct virtchnl2_rxq_info);
> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_tx_queues)) /
> + sizeof(struct virtchnl2_txq_info);
> +
> + adapter->cur_vports = 0;
> + adapter->cur_vport_nb = 0;
> +
> + return ret;
> +
> +err_vports:
> + rte_free(adapter->vport_recv_info);
> + adapter->vport_recv_info = NULL;
> +err_vport_recv_info:
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> +err_caps:
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +err_api:
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +err_mbx:
> + idpf_ctlq_deinit(hw);
> +err:
> + return -1;
> +}
> +
> +static uint16_t
> +idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
> +{
> + uint16_t vport_idx;
> + uint16_t i;
> +
> + for (i = 0; i < max_vport_nb; i++) {
> + if (!vports[i])
> + break;
> + }
> +
> + if (i == max_vport_nb)
> + vport_idx = IDPF_INVALID_VPORT_IDX;
> + else
> + vport_idx = i;
> +
> + return vport_idx;
> +}
> +
> +static int
> +idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = init_params;
> + int ret = 0;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + dev->dev_ops = &idpf_eth_dev_ops;
> + vport->adapter = adapter;
> +
> + /* for secondary processes, we don't initialise any further as primary
> + * has already done this work.
> + */
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return ret;
> +
> + dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> +
> + ret = idpf_init_vport_req_info(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
> + goto err;
> + }
> +
> + ret = idpf_vc_create_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to create vport.");
> + goto err_create_vport;
> + }
> +
> + ret = idpf_init_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vports.");
> + goto err_init_vport;
> + }
> +
> + adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
> + adapter->max_vport_nb);
> +
> + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
> + if (dev->data->mac_addrs == NULL) {
> + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
> + ret = -ENOMEM;
> + goto err_init_vport;
> + }
> +
> + rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
> + &dev->data->mac_addrs[0]);
> +
> + return 0;
> +
> +err_init_vport:
> + idpf_vc_destroy_vport(vport);
> +err_create_vport:
> + rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
> +err:
> + return ret;
> +}
> +
> +static const struct rte_pci_id pci_id_idpf_map[] = {
> + { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +struct idpf_adapter *
> +idpf_find_adapter(struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter;
> +
> + TAILQ_FOREACH(adapter, &adapter_list, next) {
> + if (!strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE))
> + return adapter;
> + }
> +
> + return NULL;
> +}
> +
> +static int
> +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter;
> + char name[RTE_ETH_NAME_MAX_LEN];
> + int i, retval;
> + bool first_probe = FALSE;
> +
> + if (!adapter_list_init) {
> + TAILQ_INIT(&adapter_list);
> + adapter_list_init = true;
> + }
> +
> + adapter = idpf_find_adapter(pci_dev);
> + if (!adapter) {
> + first_probe = TRUE;
> + adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
> + sizeof(struct idpf_adapter), 0);
> + if (!adapter) {
> + PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
> + return -1;
> + }
> +
> + retval = idpf_adapter_init(pci_dev, adapter);
> + if (retval) {
> + PMD_INIT_LOG(ERR, "Failed to init adapter.");
> + return retval;
> + }
> +
> + TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
> + }
> +
> + retval = idpf_parse_devargs(pci_dev, adapter);
> + if (retval) {
> + PMD_INIT_LOG(ERR, "Failed to parse private devargs");
> + goto err;
> + }
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + snprintf(name, sizeof(name), "idpf_%s_vport_%d",
> + pci_dev->device.name,
> + adapter->req_vports[i]);
> + retval = rte_eth_dev_create(&pci_dev->device, name,
> + sizeof(struct idpf_vport),
> + NULL, NULL, idpf_dev_init,
> + adapter);
> + if (retval)
> + PMD_DRV_LOG(ERR, "failed to create vport %d",
> + adapter->req_vports[i]);
> + }
> +
> + return 0;
> +
> +err:
> + if (first_probe) {
> + TAILQ_REMOVE(&adapter_list, adapter, next);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> + }
> + return retval;
> +}
> +
> +static void
> +idpf_adapter_rel(struct idpf_adapter *adapter)
> +{
> + struct idpf_hw *hw = &adapter->hw;
> + int i;
> +
> + idpf_ctlq_deinit(hw);
> +
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +
> + if (adapter->vport_req_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_req_info[i]);
> + adapter->vport_req_info[i] = NULL;
> + }
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> + }
> +
> + if (adapter->vport_recv_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_recv_info[i]);
> + adapter->vport_recv_info[i] = NULL;
> + }
> + rte_free(adapter->vport_recv_info);
> + adapter->vport_recv_info = NULL;
> + }
> +
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +}
> +
> +static int
> +idpf_pci_remove(struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
> + uint16_t port_id;
> +
> + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
> + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
> + rte_eth_dev_close(port_id);
> + }
> +
> + TAILQ_REMOVE(&adapter_list, adapter, next);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> +
> + return 0;
> +}
> +
> +static struct rte_pci_driver rte_idpf_pmd = {
> + .id_table = pci_id_idpf_map,
> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
> + .probe = idpf_pci_probe,
> + .remove = idpf_pci_remove,
> +};
> +
> +/**
> + * Driver initialization routine.
> + * Invoked once at EAL init time.
> + * Register itself as the [Poll Mode] Driver of PCI devices.
> + */
> +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
> +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
> +
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> new file mode 100644
> index 0000000000..c2d72fae11
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -0,0 +1,206 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_ETHDEV_H_
> +#define _IDPF_ETHDEV_H_
> +
> +#include <stdint.h>
> +#include <rte_mbuf.h>
> +#include <rte_mempool.h>
> +#include <rte_malloc.h>
> +#include <rte_spinlock.h>
> +#include <rte_ethdev.h>
> +#include <rte_kvargs.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +
> +#include "idpf_logs.h"
> +
> +#include "idpf_osdep.h"
> +#include "idpf_type.h"
> +#include "idpf_devids.h"
> +#include "idpf_lan_txrx.h"
> +#include "idpf_lan_pf_regs.h"
> +#include "virtchnl.h"
> +#include "virtchnl2.h"
> +
> +#define IDPF_MAX_VPORT_NUM 8
> +
> +#define IDPF_DEFAULT_RXQ_NUM 16
> +#define IDPF_DEFAULT_TXQ_NUM 16
Two above defines are unused. It must be added when actually
used. Please, double check all other defines below.
> +
> +#define IDPF_INVALID_VPORT_IDX 0xffff
> +#define IDPF_TXQ_PER_GRP 1
> +#define IDPF_TX_COMPLQ_PER_GRP 1
> +#define IDPF_RXQ_PER_GRP 1
> +#define IDPF_RX_BUFQ_PER_GRP 2
> +
> +#define IDPF_CTLQ_ID -1
> +#define IDPF_CTLQ_LEN 64
> +#define IDPF_DFLT_MBX_BUF_SIZE 4096
> +
> +#define IDPF_DFLT_Q_VEC_NUM 1
> +#define IDPF_DFLT_INTERVAL 16
> +
> +#define IDPF_MAX_NUM_QUEUES 256
> +#define IDPF_MIN_BUF_SIZE 1024
> +#define IDPF_MAX_FRAME_SIZE 9728
> +#define IDPF_MIN_FRAME_SIZE 14
> +
> +#define IDPF_NUM_MACADDR_MAX 64
> +
> +#define IDPF_MAX_PKT_TYPE 1024
> +
> +#define IDPF_VLAN_TAG_SIZE 4
> +#define IDPF_ETH_OVERHEAD \
> + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
> +
> +#ifndef ETH_ADDR_LEN
> +#define ETH_ADDR_LEN 6
> +#endif
> +
> +#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
> +
> +/* Message type read in virtual channel from PF */
> +enum idpf_vc_result {
> + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
> + IDPF_MSG_NON, /* Read nothing from admin queue */
> + IDPF_MSG_SYS, /* Read system msg from admin queue */
> + IDPF_MSG_CMD, /* Read async command result */
> +};
> +
> +struct idpf_chunks_info {
> + uint32_t tx_start_qid;
> + uint32_t rx_start_qid;
> +
> + uint64_t tx_qtail_start;
> + uint32_t tx_qtail_spacing;
> + uint64_t rx_qtail_start;
> + uint32_t rx_qtail_spacing;
> +};
> +
> +struct idpf_vport {
> + struct idpf_adapter *adapter; /* Backreference to associated adapter */
> + uint16_t vport_id;
> + uint16_t num_tx_q;
> + uint16_t num_rx_q;
> +
> + uint16_t max_mtu;
> + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
> +
> + uint16_t sw_idx; /* SW idx */
> +
> + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
> + uint16_t max_pkt_len; /* Maximum packet length */
> +
> + /* Chunk info */
> + struct idpf_chunks_info chunks_info;
> +
> + /* Event from ipf */
> + bool link_up;
> + uint32_t link_speed;
> +
> + uint16_t devarg_id;
> + bool stopped;
> + struct virtchnl2_vport_stats eth_stats_offset;
> +};
> +
> +struct idpf_adapter {
> + TAILQ_ENTRY(idpf_adapter) next;
> + struct idpf_hw hw;
> + char name[IDPF_ADAPTER_NAME_LEN];
> +
> + struct virtchnl_version_info virtchnl_version;
> + struct virtchnl2_get_capabilities *caps;
> +
> + volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
> + uint32_t cmd_retval; /* return value of the cmd response from ipf */
> + uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
> +
> + /* Vport info */
> + uint8_t **vport_req_info;
> + uint8_t **vport_recv_info;
> + struct idpf_vport **vports;
> + uint16_t max_vport_nb;
> + uint16_t req_vports[IDPF_MAX_VPORT_NUM];
> + uint16_t req_vport_nb;
> + uint16_t cur_vports;
> + uint16_t cur_vport_nb;
> + uint16_t cur_vport_idx;
> +
> + uint16_t used_vecs_num;
> +
> + /* Max config queue number per VC message */
> + uint32_t max_rxq_per_msg;
> + uint32_t max_txq_per_msg;
> +
> + uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
> +
> + bool stopped;
> +};
> +
> +TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
> +extern struct idpf_adapter_list adapter_list;
> +
> +#define IDPF_DEV_TO_PCI(eth_dev) \
> + RTE_DEV_TO_PCI((eth_dev)->device)
> +
> +/* structure used for sending and checking response of virtchnl ops */
> +struct idpf_cmd_info {
> + uint32_t ops;
> + uint8_t *in_args; /* buffer for sending */
> + uint32_t in_args_size; /* buffer size for sending */
> + uint8_t *out_buffer; /* buffer for response */
> + uint32_t out_size; /* buffer size for response */
> +};
> +
> +/* notify current command done. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
> +{
> + adapter->cmd_retval = msg_ret;
> + rte_wmb();
Please, add a comment which explains why memory barier is
requied here.
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
> +}
> +
> +/* clear current command. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_clear_cmd(struct idpf_adapter *adapter)
> +{
> + rte_wmb();
Please, add a comment which explains why memory barier is
requied here.
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
> + adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
> +}
> +
> +/* Check there is pending cmd in execution. If none, set new command. */
> +static inline int
> +_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
It is bad to start a symbol from single underscore, since it is
reservied for C compiler internal use.
> +{
> + enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
> + int ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
> + 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
> +
> + if (!ret)
> + PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
> +
> + return !ret;
> +}
> +
> +struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
> +int idpf_dev_link_update(struct rte_eth_dev *dev,
> + __rte_unused int wait_to_complete);
> +void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
> +int idpf_vc_check_api_version(struct idpf_adapter *adapter);
> +int idpf_vc_get_caps(struct idpf_adapter *adapter);
> +int idpf_vc_create_vport(struct rte_eth_dev *dev);
> +int idpf_vc_destroy_vport(struct idpf_vport *vport);
> +int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
> +int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
> + uint16_t buf_len, uint8_t *buf);
> +
> +#endif /* _IDPF_ETHDEV_H_ */
> diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
> new file mode 100644
> index 0000000000..a64544f86e
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_logs.h
> @@ -0,0 +1,42 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_LOGS_H_
> +#define _IDPF_LOGS_H_
> +
> +#include <rte_log.h>
> +
> +extern int idpf_logtype_init;
> +extern int idpf_logtype_driver;
> +
> +#define PMD_INIT_LOG(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
> + "%s(): " fmt "\n", __func__, ##args)
Please, use RTE_FMT/RTE_FMT_HEAD/RTE_FMT_TAIL if you
want to extend format string. See examples in many
drivers and libraries.
> +
> +#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
> +
> +#define PMD_DRV_LOG_RAW(level, fmt, args...) \
> + rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
> + "%s(): " fmt "\n", __func__, ##args)
> +
> +#define PMD_DRV_LOG(level, fmt, args...) \
> + PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
> +
> +#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
> +
> +#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
> +#define PMD_RX_LOG(level, fmt, args...) \
> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#else
> +#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
> +#endif
> +
> +#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
> +#define PMD_TX_LOG(level, fmt, args...) \
> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
> +#else
> +#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
> +#endif
> +
> +#endif /* _IDPF_LOGS_H_ */
> diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
> new file mode 100644
> index 0000000000..ef0288ff45
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_vchnl.c
> @@ -0,0 +1,495 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <stdio.h>
> +#include <errno.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <stdarg.h>
> +#include <inttypes.h>
> +#include <rte_byteorder.h>
> +#include <rte_common.h>
> +
> +#include <rte_debug.h>
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#include "idpf_prototype.h"
> +
> +#define IDPF_CTLQ_LEN 64
> +
> +static int
> +idpf_vc_clean(struct idpf_adapter *adapter)
> +{
> + struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
> + uint16_t num_q_msg = IDPF_CTLQ_LEN;
> + struct idpf_dma_mem *dma_mem;
> + int err = 0;
> + uint32_t i;
> +
> + for (i = 0; i < 10; i++) {
> + err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
> + msleep(20);
> + if (num_q_msg)
> + break;
> + }
> + if (err)
> + goto error;
Why not just return err? Be consistent since you do not use
goto to handle errors below.
> +
> + /* Empty queue is not an error */
> + for (i = 0; i < num_q_msg; i++) {
> + dma_mem = q_msg[i]->ctx.indirect.payload;
> + if (dma_mem) {
> + idpf_free_dma_mem(&adapter->hw, dma_mem);
> + rte_free(dma_mem);
> + }
> + rte_free(q_msg[i]);
> + }
> +
> +error:
> + return err;
> +}
> +
> +static int
> +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
> + uint16_t msg_size, uint8_t *msg)
> +{
> + struct idpf_ctlq_msg *ctlq_msg;
> + struct idpf_dma_mem *dma_mem;
> + int err = 0;
> +
> + err = idpf_vc_clean(adapter);
> + if (err)
> + goto err;
> +
> + ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
> + sizeof(struct idpf_ctlq_msg), 0);
> + if (!ctlq_msg) {
> + err = -ENOMEM;
> + goto err;
> + }
> +
> + dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
> + sizeof(struct idpf_dma_mem), 0);
> + if (!dma_mem) {
> + err = -ENOMEM;
> + goto dma_mem_error;
> + }
> +
> + dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
> + idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
> + if (!dma_mem->va) {
> + err = -ENOMEM;
> + goto dma_alloc_error;
> + }
> +
> + memcpy(dma_mem->va, msg, msg_size);
> +
> + ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
> + ctlq_msg->func_id = 0;
> + ctlq_msg->data_len = msg_size;
> + ctlq_msg->cookie.mbx.chnl_opcode = op;
> + ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
> + ctlq_msg->ctx.indirect.payload = dma_mem;
> +
> + err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
> + if (err)
> + goto send_error;
> +
> + return err;
just return 0 to make it easier to read.
> +
> +send_error:
> + idpf_free_dma_mem(&adapter->hw, dma_mem);
> +dma_alloc_error:
> + rte_free(dma_mem);
> +dma_mem_error:
> + rte_free(ctlq_msg);
> +err:
> + return err;
> +}
> +
> +static enum idpf_vc_result
> +idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
> + uint8_t *buf)
> +{
> + struct idpf_hw *hw = &adapter->hw;
> + struct idpf_ctlq_msg ctlq_msg;
> + struct idpf_dma_mem *dma_mem = NULL;
> + enum idpf_vc_result result = IDPF_MSG_NON;
> + enum virtchnl_ops opcode;
> + uint16_t pending = 1;
> + int ret;
> +
> + ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
> + if (ret) {
> + PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
> + if (ret != IDPF_ERR_CTLQ_NO_WORK)
> + result = IDPF_MSG_ERR;
> + return result;
> + }
> +
> + rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
> +
> + opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
> + adapter->cmd_retval =
> + (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
> +
> + PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
> + opcode, adapter->cmd_retval);
> +
> + if (opcode == VIRTCHNL2_OP_EVENT) {
> + struct virtchnl2_event *ve =
> + (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
> +
> + result = IDPF_MSG_SYS;
> + switch (ve->event) {
> + case VIRTCHNL2_EVENT_LINK_CHANGE:
> + /* TBD */
> + break;
> + default:
> + PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
> + __func__, ve->event);
> + break;
> + }
> + } else {
> + /* async reply msg on command issued by pf previously */
> + result = IDPF_MSG_CMD;
> + if (opcode != adapter->pend_cmd) {
> + PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
> + adapter->pend_cmd, opcode);
> + result = IDPF_MSG_ERR;
> + }
> + }
> +
> + if (ctlq_msg.data_len)
> + dma_mem = ctlq_msg.ctx.indirect.payload;
> + else
> + pending = 0;
> +
> + ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
> + if (ret && dma_mem)
> + idpf_free_dma_mem(hw, dma_mem);
> +
> + return result;
> +}
> +
> +#define MAX_TRY_TIMES 200
> +#define ASQ_DELAY_MS 10
> +
> +int
> +idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
> + uint8_t *buf)
> +{
> + int ret, err = 0, i = 0;
It is very error prone to declare and initialize varialbes in
the same line. Not a requirement, but just an advice to avoid
it and have own line for each variable as you do below.
> +
> + do {
> + ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
> + if (ret == IDPF_MSG_CMD)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + } while (i++ < MAX_TRY_TIMES);
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
> + adapter->cmd_retval, ops);
> + }
> +
> + return err;
> +}
> +
> +static int
> +idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
> +{
> + int err = 0;
> + int i = 0;
> + int ret;
> +
> + if (_atomic_set_cmd(adapter, args->ops))
> + return -1;
> +
> + ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
> + if (ret) {
> + PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
> + _clear_cmd(adapter);
> + return ret;
> + }
> +
> + switch (args->ops) {
> + case VIRTCHNL_OP_VERSION:
> + case VIRTCHNL2_OP_GET_CAPS:
> + case VIRTCHNL2_OP_CREATE_VPORT:
> + case VIRTCHNL2_OP_DESTROY_VPORT:
> + case VIRTCHNL2_OP_SET_RSS_KEY:
> + case VIRTCHNL2_OP_SET_RSS_LUT:
> + case VIRTCHNL2_OP_SET_RSS_HASH:
> + case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
> + case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_QUEUES:
> + case VIRTCHNL2_OP_DISABLE_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_VPORT:
> + case VIRTCHNL2_OP_DISABLE_VPORT:
> + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
> + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
> + case VIRTCHNL2_OP_ALLOC_VECTORS:
> + case VIRTCHNL2_OP_DEALLOC_VECTORS:
> + case VIRTCHNL2_OP_GET_STATS:
> + /* for init virtchnl ops, need to poll the response */
> + err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
> + _clear_cmd(adapter);
> + break;
> + case VIRTCHNL2_OP_GET_PTYPE_INFO:
> + /* for multuple response message,
> + * do not handle the response here.
> + */
> + break;
> + default:
> + /* For other virtchnl ops in running time,
> + * wait for the cmd done flag.
> + */
> + do {
> + if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + /* If don't read msg or read sys event, continue */
> + } while (i++ < MAX_TRY_TIMES);
> + /* If there's no response is received, clear command */
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
> + adapter->cmd_retval, args->ops);
> + _clear_cmd(adapter);
> + }
> + break;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_vc_check_api_version(struct idpf_adapter *adapter)
> +{
> + struct virtchnl_version_info version;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&version, 0, sizeof(struct virtchnl_version_info));
> + version.major = VIRTCHNL_VERSION_MAJOR_2;
> + version.minor = VIRTCHNL_VERSION_MINOR_0;
> +
> + args.ops = VIRTCHNL_OP_VERSION;
> + args.in_args = (uint8_t *)&version;
> + args.in_args_size = sizeof(version);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of VIRTCHNL_OP_VERSION");
> + return err;
> + }
> +
> + return err;
Just return 0 here or no return err above.
> +}
> +
> +int
> +idpf_vc_get_caps(struct idpf_adapter *adapter)
> +{
> + struct virtchnl2_get_capabilities caps_msg;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> + caps_msg.csum_caps =
> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> +
> + caps_msg.seg_caps =
> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> + VIRTCHNL2_CAP_SEG_GENERIC;
> +
> + caps_msg.rss_caps =
> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> +
> + caps_msg.hsplit_caps =
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
> +
> + caps_msg.rsc_caps =
> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
> +
> + caps_msg.other_caps =
> + VIRTCHNL2_CAP_RDMA |
> + VIRTCHNL2_CAP_SRIOV |
> + VIRTCHNL2_CAP_MACFILTER |
> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> + VIRTCHNL2_CAP_CRC |
> + VIRTCHNL2_CAP_WB_ON_ITR |
> + VIRTCHNL2_CAP_PROMISC |
> + VIRTCHNL2_CAP_LINK_SPEED |
> + VIRTCHNL2_CAP_VLAN;
> +
> + args.ops = VIRTCHNL2_OP_GET_CAPS;
> + args.in_args = (uint8_t *)&caps_msg;
> + args.in_args_size = sizeof(caps_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
> + return err;
> + }
> +
> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> +
> + return err;
Just return 0 to make it easier to read.
> +}
> +
> +int
> +idpf_vc_create_vport(struct rte_eth_dev *dev)
In general, the function does not look ethdev specific.
If you have different classes of devices in the future,
most likely you'll need the same code.
Anyway I'd consider to make the prototype not ethdev
specific and not PCI specific anyway. It looks like
intput could be just 'struct idpf_adapter *'.
It is better to keep all ethdev interface specifics in
idpf_ethdev.c. Of course, it is not always possible, but
it would make next-net maintainers life easier since it
will limit scope of attentive review.
> +{
> + struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
> + uint16_t idx = adapter->cur_vport_idx;
> + struct virtchnl2_create_vport *vport_req_info =
> + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
> + struct virtchnl2_create_vport vport_msg;
> + struct idpf_cmd_info args;
> + int err = -1;
> +
> + memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
> + vport_msg.vport_type = vport_req_info->vport_type;
> + vport_msg.txq_model = vport_req_info->txq_model;
> + vport_msg.rxq_model = vport_req_info->rxq_model;
> + vport_msg.num_tx_q = vport_req_info->num_tx_q;
> + vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
> + vport_msg.num_rx_q = vport_req_info->num_rx_q;
> + vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_CREATE_VPORT;
> + args.in_args = (uint8_t *)&vport_msg;
> + args.in_args_size = sizeof(vport_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
> + return err;
> + }
> +
> + if (!adapter->vport_recv_info[idx]) {
> + adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
> + IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->vport_recv_info[idx]) {
> + PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
> + return err;
You will return 0 here. It is better to be explicit and return
just -1 instead of usage helper variable.
> + }
> + }
> + rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
> + IDPF_DFLT_MBX_BUF_SIZE);
> + return err;
just return 0.
> +}
> +
> +int
> +idpf_vc_destroy_vport(struct idpf_vport *vport)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
> + args.in_args = (uint8_t *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
> + return err;
> + }
> +
> + return err;
If you have return above, it would be easier to read
if you return 0 here.
> +}
> +
> +int
> +idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
> + VIRTCHNL2_OP_DISABLE_VPORT;
> + args.in_args = (u8 *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
> + enable ? "ENABLE" : "DISABLE");
> + }
> +
> + return err;
Please, be consistent. You have return in if branch in the
function above, but od not have return in this function
whereas code pattern is absolutely the same.
> +}
> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> new file mode 100644
> index 0000000000..bf8bf58ef5
> --- /dev/null
> +++ b/drivers/net/idpf/meson.build
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 Intel Corporation
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
> +endif
> +
> +includes += include_directories('../../common/idpf')
Isn't common dependency insufficient to have it in includes path?
> +deps += ['common_idpf', 'security', 'cryptodev']
> +
> +sources = files(
> + 'idpf_ethdev.c',
> + 'idpf_vchnl.c',
> +)
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model
2022-10-21 5:18 ` [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
@ 2022-10-21 7:44 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 7:44 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
Just top level review for the rest of patches starting from
this one.
On 10/21/22 08:18, Junfeng Guo wrote:
> Add support for queue operations in single queue model:
> - rx_queue_setup
> - rx_queue_release
> - tx_queue_setup
> - tx_queue_release
>
> In the single queue model, the same descriptor queue is used by SW to
> post buffer descriptors to HW and by HW to post completed descriptors
> to SW.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> doc/guides/nics/features/idpf.ini | 2 +
> doc/guides/nics/idpf.rst | 22 ++
> drivers/net/idpf/idpf_ethdev.c | 58 ++++
> drivers/net/idpf/idpf_ethdev.h | 9 +
> drivers/net/idpf/idpf_rxtx.c | 465 ++++++++++++++++++++++++++++++
> drivers/net/idpf/idpf_rxtx.h | 186 ++++++++++++
> drivers/net/idpf/idpf_vchnl.c | 251 ++++++++++++++++
> drivers/net/idpf/meson.build | 1 +
> 8 files changed, 994 insertions(+)
> create mode 100644 drivers/net/idpf/idpf_rxtx.c
> create mode 100644 drivers/net/idpf/idpf_rxtx.h
>
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> index f029a279b3..681a908194 100644
> --- a/doc/guides/nics/features/idpf.ini
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -7,6 +7,8 @@
> ; is selected.
> ;
> [Features]
> +Runtime Rx queue setup = Y
> +Runtime Tx queue setup = Y
It does not make sense when device does not support start yet.
Typically it requies extra code to support runtime queues
setup. So, it is better to add the feature in a separate patch
after device start support.
> Multiprocess aware = Y
> FreeBSD = Y
> Linux = Y
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 02/14] net/idpf: add support for device initialization
2022-10-21 7:39 ` Andrew Rybchenko
@ 2022-10-21 7:48 ` Andrew Rybchenko
2022-10-21 12:41 ` Zhang, Qi Z
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 7:48 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang
On 10/21/22 10:39, Andrew Rybchenko wrote:
> On 10/21/22 08:18, Junfeng Guo wrote:
>> Support device init and add the following dev ops skeleton:
>> - dev_configure
>> - dev_start
One more question: are you sure that you can start without
queues setup?
>> - dev_stop
>> - dev_close
>>
>> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
>> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
>> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
>> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>> ---
>> MAINTAINERS | 9 +
>> doc/guides/nics/features/idpf.ini | 15 +
>> doc/guides/nics/idpf.rst | 52 ++
>> doc/guides/nics/index.rst | 1 +
>> doc/guides/rel_notes/release_22_11.rst | 6 +
>> drivers/net/idpf/idpf_ethdev.c | 774 +++++++++++++++++++++++++
>> drivers/net/idpf/idpf_ethdev.h | 206 +++++++
>> drivers/net/idpf/idpf_logs.h | 42 ++
>> drivers/net/idpf/idpf_vchnl.c | 495 ++++++++++++++++
>> drivers/net/idpf/meson.build | 16 +
>> drivers/net/idpf/version.map | 3 +
>> drivers/net/meson.build | 1 +
>> 12 files changed, 1620 insertions(+)
>> create mode 100644 doc/guides/nics/features/idpf.ini
>> create mode 100644 doc/guides/nics/idpf.rst
>> create mode 100644 drivers/net/idpf/idpf_ethdev.c
>> create mode 100644 drivers/net/idpf/idpf_ethdev.h
>> create mode 100644 drivers/net/idpf/idpf_logs.h
>> create mode 100644 drivers/net/idpf/idpf_vchnl.c
>> create mode 100644 drivers/net/idpf/meson.build
>> create mode 100644 drivers/net/idpf/version.map
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 92b381bc30..34f8b9cc61 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -764,6 +764,15 @@ F: drivers/net/ice/
>> F: doc/guides/nics/ice.rst
>> F: doc/guides/nics/features/ice.ini
>> +Intel idpf
>> +M: Jingjing Wu <jingjing.wu@intel.com>
>> +M: Beilei Xing <beilei.xing@intel.com>
>> +T: git://dpdk.org/next/dpdk-next-net-intel
>> +F: drivers/net/idpf/
>> +F: drivers/common/idpf/
>> +F: doc/guides/nics/idpf.rst
>> +F: doc/guides/nics/features/idpf.ini
>> +
>> Intel igc
>> M: Junfeng Guo <junfeng.guo@intel.com>
>> M: Simei Su <simei.su@intel.com>
>> diff --git a/doc/guides/nics/features/idpf.ini
>> b/doc/guides/nics/features/idpf.ini
>> new file mode 100644
>> index 0000000000..f029a279b3
>> --- /dev/null
>> +++ b/doc/guides/nics/features/idpf.ini
>> @@ -0,0 +1,15 @@
>> +;
>> +; Supported features of the 'idpf' network poll mode driver.
>> +;
>> +; Refer to default.ini for the full list of available PMD features.
>> +;
>> +; A feature with "P" indicates only be supported when non-vector path
>> +; is selected.
>> +;
>> +[Features]
>> +Multiprocess aware = Y
>
> Is multi-process support tested?
>
>> +FreeBSD = Y
>
> Have you actually checked it on FreeBSD?
>
>> +Linux = Y
>> +Windows = Y
>
> It looks ridiculous taking into account that meson.build says
> that "not supported on Windows".
>
>> +x86-32 = Y
>> +x86-64 = Y
>> diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
>> new file mode 100644
>> index 0000000000..428bf4266a
>> --- /dev/null
>> +++ b/doc/guides/nics/idpf.rst
>> @@ -0,0 +1,52 @@
>> +.. SPDX-License-Identifier: BSD-3-Clause
>> + Copyright(c) 2022 Intel Corporation.
>> +
>> +IDPF Poll Mode Driver
>> +======================
>> +
>> +The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll
>> mode driver support for
>> +Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
>> +
>
> Coding style requires two empty lines before the next section.
> Here and everywhere below.
>
>> +Linux Prerequisites
>> +-------------------
>> +
>> +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>`
>> to setup the basic DPDK environment.
>> +
>> +- To get better performance on Intel platforms, please follow the
>> "How to get best performance with NICs on Intel platforms"
>> + section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
>> +
>> +Windows Prerequisites
>> +---------------------
>
> Sorry, it does not make sense since build on Windows is not
> supported.
>
>> +
>> +- Follow the :doc:`guide for Windows <../windows_gsg/run_apps>`
>> + to setup the basic DPDK environment.
>> +
>> +- Identify the Intel® Ethernet adapter and get the latest NVM/FW
>> version.
>> +
>> +- To access any Intel® Ethernet hardware, load the NetUIO driver in
>> place of existing built-in (inbox) driver.
>> +
>> +- To load NetUIO driver, follow the steps mentioned in `dpdk-kmods
>> repository
>> + <https://git.dpdk.org/dpdk-kmods/tree/windows/netuio/README.rst>`_.
>> +
>> +Pre-Installation Configuration
>> +------------------------------
>> +
>> +Runtime Config Options
>> +~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +- ``vport`` (default ``not create ethdev``)
>> +
>> + The IDPF PMD supports creation of multiple vports for one PCI
>> device, each vport
>> + corresponds to a single ethdev. Using the ``devargs`` parameter
>> ``vport`` the user
>> + can specify the vports with specific ID to be created, for example::
>> +
>> + -a ca:00.0,vport=[0,2,3]
>> +
>> + Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
>> + NOTE: This parameter is MUST, otherwise there'll be no any ethdev
>> created.
>
> It hardly user-friendly. Of course, you know better, but I'd
> consider to have some sensible default which is not NOP.
>
>> +
>> +Driver compilation and testing
>> +------------------------------
>> +
>> +Refer to the document :ref:`compiling and testing a PMD for a NIC
>> <pmd_build_and_test>`
>> +for details.
>
> [snip]
>
>> diff --git a/drivers/net/idpf/idpf_ethdev.c
>> b/drivers/net/idpf/idpf_ethdev.c
>> new file mode 100644
>> index 0000000000..7806c43668
>> --- /dev/null
>> +++ b/drivers/net/idpf/idpf_ethdev.c
>> @@ -0,0 +1,774 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2022 Intel Corporation
>> + */
>> +
>> +#include <rte_atomic.h>
>> +#include <rte_eal.h>
>> +#include <rte_ether.h>
>> +#include <rte_malloc.h>
>> +#include <rte_memzone.h>
>> +#include <rte_dev.h>
>> +
>> +#include "idpf_ethdev.h"
>> +
>> +#define IDPF_VPORT "vport"
>> +
>> +struct idpf_adapter_list adapter_list;
>> +bool adapter_list_init;
>
> If it is global, it should be exmplained what it is and why
> locking is not required.
>
>> +
>> +static const char * const idpf_valid_args[] = {
>> + IDPF_VPORT,
>> + NULL
>> +};
>> +
>> +static int idpf_dev_configure(struct rte_eth_dev *dev);
>> +static int idpf_dev_start(struct rte_eth_dev *dev);
>> +static int idpf_dev_stop(struct rte_eth_dev *dev);
>> +static int idpf_dev_close(struct rte_eth_dev *dev);
>> +static void idpf_adapter_rel(struct idpf_adapter *adapter);
>> +
>> +int
>> +idpf_dev_link_update(struct rte_eth_dev *dev,
>> + __rte_unused int wait_to_complete)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> + struct rte_eth_link new_link;
>> +
>> + memset(&new_link, 0, sizeof(new_link));
>> +
>> + new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
>> +
>> + new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
>> + new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
>> + RTE_ETH_LINK_DOWN;
>> + new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
>> + RTE_ETH_LINK_SPEED_FIXED);
>> +
>> + return rte_eth_linkstatus_set(dev, &new_link);
>> +}
>> +
>> +static const struct eth_dev_ops idpf_eth_dev_ops = {
>> + .dev_configure = idpf_dev_configure,
>> + .dev_start = idpf_dev_start,
>> + .dev_stop = idpf_dev_stop,
>> + .dev_close = idpf_dev_close,
>> + .link_update = idpf_dev_link_update,
>> +};
>> +
>> +static int
>> +idpf_init_vport_req_info(struct rte_eth_dev *dev)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> + struct idpf_adapter *adapter = vport->adapter;
>> + struct virtchnl2_create_vport *vport_info;
>> + uint16_t idx = adapter->cur_vport_idx;
>> +
>> + if (idx == IDPF_INVALID_VPORT_IDX) {
>> + PMD_INIT_LOG(ERR, "Invalid vport index.");
>> + return -1;
>> + }
>> +
>> + if (!adapter->vport_req_info[idx]) {
>
> Compare vs NULL as DPDK coding style says.
>
>> + adapter->vport_req_info[idx] = rte_zmalloc(NULL,
>> + sizeof(struct virtchnl2_create_vport), 0);
>> + if (!adapter->vport_req_info[idx]) {
>
> Compare vs NULL. Please, review the code yourself
> since otherwise we can play ping-pong for a long-long
> time.
>
>> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
>> + return -1;
>> + }
>> + }
>> +
>> + vport_info =
>> + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
>> +
>> + vport_info->vport_type =
>> rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
>> +
>> + return 0;
>> +}
>> +
>> +static uint16_t
>> +idpf_parse_devarg_id(char *name)
>> +{
>> + uint16_t val;
>> + char *p;
>> +
>> + p = strstr(name, "vport_");
>> + p += sizeof("vport_") - 1;
>> +
>> + val = strtoul(p, NULL, 10);
>> +
>> + return val;
>> +}
>> +
>> +static int
>> +idpf_init_vport(struct rte_eth_dev *dev)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> + struct idpf_adapter *adapter = vport->adapter;
>> + uint16_t idx = adapter->cur_vport_idx;
>> + struct virtchnl2_create_vport *vport_info =
>> + (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
>> + int i;
>> +
>> + vport->vport_id = vport_info->vport_id;
>> + vport->num_tx_q = vport_info->num_tx_q;
>> + vport->num_rx_q = vport_info->num_rx_q;
>> + vport->max_mtu = vport_info->max_mtu;
>> + rte_memcpy(vport->default_mac_addr,
>> + vport_info->default_mac_addr, ETH_ALEN);
>> + vport->sw_idx = idx;
>> +
>> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
>> + if (vport_info->chunks.chunks[i].type ==
>> + VIRTCHNL2_QUEUE_TYPE_TX) {
>> + vport->chunks_info.tx_start_qid =
>> + vport_info->chunks.chunks[i].start_queue_id;
>> + vport->chunks_info.tx_qtail_start =
>> + vport_info->chunks.chunks[i].qtail_reg_start;
>> + vport->chunks_info.tx_qtail_spacing =
>> + vport_info->chunks.chunks[i].qtail_reg_spacing;
>> + } else if (vport_info->chunks.chunks[i].type ==
>> + VIRTCHNL2_QUEUE_TYPE_RX) {
>> + vport->chunks_info.rx_start_qid =
>> + vport_info->chunks.chunks[i].start_queue_id;
>> + vport->chunks_info.rx_qtail_start =
>> + vport_info->chunks.chunks[i].qtail_reg_start;
>> + vport->chunks_info.rx_qtail_spacing =
>> + vport_info->chunks.chunks[i].qtail_reg_spacing;
>> + }
>> + }
>> +
>> + vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
>> + vport->dev_data = dev->data;
>> + vport->stopped = 1;
>> +
>> + adapter->vports[idx] = vport;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
>> +{
>
> Sorry, but you need to check configuration here and reject
> unsupported configuration options which are not rejected by
> ethdev layer.
>
>> + return 0;
>> +}
>> +
>> +static int
>> +idpf_dev_start(struct rte_eth_dev *dev)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> +
>> + PMD_INIT_FUNC_TRACE();
>
> Can we call it in a secondary process? Will it work?
>
>> +
>> + vport->stopped = 0;
>> +
>> + if (idpf_vc_ena_dis_vport(vport, true)) {
>
> Compare vs 0 since return value is not bool.
> Please, review all conditions yourself.
>
>> + PMD_DRV_LOG(ERR, "Failed to enable vport");
>> + goto err_vport;
>> + }
>> +
>> + return 0;
>> +
>> +err_vport:
>> + return -1;
>> +}
>> +
>> +static int
>> +idpf_dev_stop(struct rte_eth_dev *dev)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> +
>> + PMD_INIT_FUNC_TRACE();
>
> What about secondary process? Will it work?
>
>> +
>> + if (vport->stopped == 1)
>
> In therory, rte_eth_dev_stop() cares about it. If we need to
> care about it here as well, it should be explaiend in a comment
> why.
>
>> + return 0;
>> +
>> + if (idpf_vc_ena_dis_vport(vport, false))
>> + PMD_DRV_LOG(ERR, "disable vport failed");
>
> Don't you want to propagate the failure further
> (since operation can return status)?
> Otherwise, it should be clear in logs that the failure is
> ignored.
>
>> +
>> + vport->stopped = 1;
>> + dev->data->dev_started = 0;
>
> rte_eth_dev_stop() does it. So, it is not necessary here.
>
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +idpf_dev_close(struct rte_eth_dev *dev)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> + struct idpf_adapter *adapter = vport->adapter;
>> +
>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>> + return 0;
>> +
>> + idpf_dev_stop(dev);
>> + idpf_vc_destroy_vport(vport);
>> +
>> + adapter->cur_vports &= ~BIT(vport->devarg_id);
>
> Why not RTE_BIT32()?
>
>> +
>> + rte_free(vport);
>> + dev->data->dev_private = NULL;
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +insert_value(struct idpf_adapter *adapter, uint16_t id)
>> +{
>> + uint16_t i;
>> +
>> + for (i = 0; i < adapter->req_vport_nb; i++) {
>> + if (adapter->req_vports[i] == id)
>> + return 0;
>> + }
>> +
>> + if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
>> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
>> + IDPF_MAX_VPORT_NUM);
>> + return -1;
>> + }
>> +
>> + adapter->req_vports[adapter->req_vport_nb] = id;
>> + adapter->req_vport_nb++;
>> +
>> + return 0;
>> +}
>> +
>> +static const char *
>> +parse_range(const char *value, struct idpf_adapter *adapter)
>> +{
>> + uint16_t lo, hi, i;
>> + int n = 0;
>> + int result;
>> + const char *pos = value;
>> +
>> + result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
>> + if (result == 1) {
>> + if (lo >= IDPF_MAX_VPORT_NUM)
>> + return NULL;
>> + if (insert_value(adapter, lo))
>> + return NULL;
>> + } else if (result == 2) {
>> + if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
>> + return NULL;
>> + for (i = lo; i <= hi; i++) {
>> + if (insert_value(adapter, i))
>> + return NULL;
>> + }
>> + } else {
>> + return NULL;
>> + }
>> +
>> + return pos + n;
>> +}
>> +
>> +static int
>> +parse_vport(const char *key, const char *value, void *args)
>> +{
>> + struct idpf_adapter *adapter = (struct idpf_adapter *)args;
>> + const char *pos = value;
>> + int i;
>> +
>> + adapter->req_vport_nb = 0;
>> +
>> + if (*pos == '[')
>> + pos++;
>> +
>> + while (1) {
>> + pos = parse_range(pos, adapter);
>> + if (pos == NULL) {
>> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
>> + value, key);
>> + return -1;
>> + }
>> + if (*pos != ',')
>> + break;
>> + pos++;
>> + }
>> +
>> + if (*value == '[' && *pos != ']') {
>> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
>> + value, key);
>> + return -1;
>> + }
>> +
>> + if (adapter->cur_vport_nb + adapter->req_vport_nb >
>> + IDPF_MAX_VPORT_NUM) {
>> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
>> + IDPF_MAX_VPORT_NUM);
>> + return -1;
>> + }
>> +
>> + for (i = 0; i < adapter->req_vport_nb; i++) {
>> + if (!(adapter->cur_vports & BIT(adapter->req_vports[i]))) {
>> + adapter->cur_vports |= BIT(adapter->req_vports[i]);
>> + adapter->cur_vport_nb++;
>> + } else {
>> + PMD_INIT_LOG(ERR, "Vport %d has been created",
>> + adapter->req_vports[i]);
>> + return -1;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int
>> +idpf_parse_devargs(struct rte_pci_device *pci_dev, struct
>> idpf_adapter *adapter)
>> +{
>> + struct rte_devargs *devargs = pci_dev->device.devargs;
>> + struct rte_kvargs *kvlist;
>> + int ret;
>> +
>> + if (!devargs)
>> + return 0;
>> +
>> + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
>> + if (!kvlist) {
>> + PMD_INIT_LOG(ERR, "invalid kvargs key");
>> + return -EINVAL;
>> + }
>> +
>> + ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
>> + adapter);
>> + if (ret)
>> + goto bail;
>> +
>> +bail:
>> + rte_kvargs_free(kvlist);
>> + return ret;
>> +}
>> +
>> +static void
>> +idpf_reset_pf(struct idpf_hw *hw)
>> +{
>> + uint32_t reg;
>> +
>> + reg = IDPF_READ_REG(hw, PFGEN_CTRL);
>> + IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
>> +}
>> +
>> +#define IDPF_RESET_WAIT_CNT 100
>> +static int
>> +idpf_check_pf_reset_done(struct idpf_hw *hw)
>> +{
>> + uint32_t reg;
>> + int i;
>> +
>> + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
>> + reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
>> + if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
>> + return 0;
>> + rte_delay_ms(1000);
>> + }
>> +
>> + PMD_INIT_LOG(ERR, "IDPF reset timeout");
>> + return -EBUSY;
>> +}
>> +
>> +#define CTLQ_NUM 2
>> +static int
>> +idpf_init_mbx(struct idpf_hw *hw)
>> +{
>> + struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
>> + {
>> + .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
>> + .id = IDPF_CTLQ_ID,
>> + .len = IDPF_CTLQ_LEN,
>> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
>> + .reg = {
>> + .head = PF_FW_ATQH,
>> + .tail = PF_FW_ATQT,
>> + .len = PF_FW_ATQLEN,
>> + .bah = PF_FW_ATQBAH,
>> + .bal = PF_FW_ATQBAL,
>> + .len_mask = PF_FW_ATQLEN_ATQLEN_M,
>> + .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
>> + .head_mask = PF_FW_ATQH_ATQH_M,
>> + }
>> + },
>> + {
>> + .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
>> + .id = IDPF_CTLQ_ID,
>> + .len = IDPF_CTLQ_LEN,
>> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
>> + .reg = {
>> + .head = PF_FW_ARQH,
>> + .tail = PF_FW_ARQT,
>> + .len = PF_FW_ARQLEN,
>> + .bah = PF_FW_ARQBAH,
>> + .bal = PF_FW_ARQBAL,
>> + .len_mask = PF_FW_ARQLEN_ARQLEN_M,
>> + .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
>> + .head_mask = PF_FW_ARQH_ARQH_M,
>> + }
>> + }
>> + };
>> + struct idpf_ctlq_info *ctlq;
>> + int ret;
>> +
>> + ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
>> + if (ret)
>> + return ret;
>> +
>> + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
>> + struct idpf_ctlq_info, cq_list) {
>> + if (ctlq->q_id == IDPF_CTLQ_ID &&
>> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
>> + hw->asq = ctlq;
>> + if (ctlq->q_id == IDPF_CTLQ_ID &&
>> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
>> + hw->arq = ctlq;
>> + }
>> +
>> + if (!hw->asq || !hw->arq) {
>> + idpf_ctlq_deinit(hw);
>> + ret = -ENOENT;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int
>> +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter
>> *adapter)
>> +{
>> + struct idpf_hw *hw = &adapter->hw;
>> + int ret = 0;
>> +
>> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
>> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
>> + hw->back = adapter;
>> + hw->vendor_id = pci_dev->id.vendor_id;
>> + hw->device_id = pci_dev->id.device_id;
>> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
>> +
>> + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
>> +
>> + idpf_reset_pf(hw);
>> + ret = idpf_check_pf_reset_done(hw);
>> + if (ret) {
>> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
>> + goto err;
>> + }
>> +
>> + ret = idpf_init_mbx(hw);
>> + if (ret) {
>> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
>> + goto err;
>> + }
>> +
>> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
>> + IDPF_DFLT_MBX_BUF_SIZE, 0);
>> + if (!adapter->mbx_resp) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp
>> memory");
>> + goto err_mbx;
>> + }
>> +
>> + if (idpf_vc_check_api_version(adapter)) {
>> + PMD_INIT_LOG(ERR, "Failed to check api version");
>> + goto err_api;
>> + }
>> +
>> + adapter->caps = rte_zmalloc("idpf_caps",
>> + sizeof(struct virtchnl2_get_capabilities), 0);
>> + if (!adapter->caps) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
>> + goto err_api;
>> + }
>> +
>> + if (idpf_vc_get_caps(adapter)) {
>> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
>> + goto err_caps;
>> + }
>> +
>> + adapter->max_vport_nb = adapter->caps->max_vports;
>> +
>> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
>> + adapter->max_vport_nb *
>> + sizeof(*adapter->vport_req_info),
>> + 0);
>> + if (!adapter->vport_req_info) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
>> + goto err_caps;
>> + }
>> +
>> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
>> + adapter->max_vport_nb *
>> + sizeof(*adapter->vport_recv_info),
>> + 0);
>> + if (!adapter->vport_recv_info) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
>> + goto err_vport_recv_info;
>> + }
>> +
>> + adapter->vports = rte_zmalloc("vports",
>> + adapter->max_vport_nb *
>> + sizeof(*adapter->vports),
>> + 0);
>> + if (!adapter->vports) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
>> + goto err_vports;
>> + }
>> +
>> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
>> + sizeof(struct virtchnl2_config_rx_queues)) /
>> + sizeof(struct virtchnl2_rxq_info);
>> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
>> + sizeof(struct virtchnl2_config_tx_queues)) /
>> + sizeof(struct virtchnl2_txq_info);
>> +
>> + adapter->cur_vports = 0;
>> + adapter->cur_vport_nb = 0;
>> +
>> + return ret;
>> +
>> +err_vports:
>> + rte_free(adapter->vport_recv_info);
>> + adapter->vport_recv_info = NULL;
>> +err_vport_recv_info:
>> + rte_free(adapter->vport_req_info);
>> + adapter->vport_req_info = NULL;
>> +err_caps:
>> + rte_free(adapter->caps);
>> + adapter->caps = NULL;
>> +err_api:
>> + rte_free(adapter->mbx_resp);
>> + adapter->mbx_resp = NULL;
>> +err_mbx:
>> + idpf_ctlq_deinit(hw);
>> +err:
>> + return -1;
>> +}
>> +
>> +static uint16_t
>> +idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
>> +{
>> + uint16_t vport_idx;
>> + uint16_t i;
>> +
>> + for (i = 0; i < max_vport_nb; i++) {
>> + if (!vports[i])
>> + break;
>> + }
>> +
>> + if (i == max_vport_nb)
>> + vport_idx = IDPF_INVALID_VPORT_IDX;
>> + else
>> + vport_idx = i;
>> +
>> + return vport_idx;
>> +}
>> +
>> +static int
>> +idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
>> +{
>> + struct idpf_vport *vport = dev->data->dev_private;
>> + struct idpf_adapter *adapter = init_params;
>> + int ret = 0;
>> +
>> + PMD_INIT_FUNC_TRACE();
>> +
>> + dev->dev_ops = &idpf_eth_dev_ops;
>> + vport->adapter = adapter;
>> +
>> + /* for secondary processes, we don't initialise any further as
>> primary
>> + * has already done this work.
>> + */
>> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
>> + return ret;
>> +
>> + dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
>> +
>> + ret = idpf_init_vport_req_info(dev);
>> + if (ret) {
>> + PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
>> + goto err;
>> + }
>> +
>> + ret = idpf_vc_create_vport(dev);
>> + if (ret) {
>> + PMD_INIT_LOG(ERR, "Failed to create vport.");
>> + goto err_create_vport;
>> + }
>> +
>> + ret = idpf_init_vport(dev);
>> + if (ret) {
>> + PMD_INIT_LOG(ERR, "Failed to init vports.");
>> + goto err_init_vport;
>> + }
>> +
>> + adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
>> + adapter->max_vport_nb);
>> +
>> + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
>> + if (dev->data->mac_addrs == NULL) {
>> + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
>> + ret = -ENOMEM;
>> + goto err_init_vport;
>> + }
>> +
>> + rte_ether_addr_copy((struct rte_ether_addr
>> *)vport->default_mac_addr,
>> + &dev->data->mac_addrs[0]);
>> +
>> + return 0;
>> +
>> +err_init_vport:
>> + idpf_vc_destroy_vport(vport);
>> +err_create_vport:
>> +
>> rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
>> +err:
>> + return ret;
>> +}
>> +
>> +static const struct rte_pci_id pci_id_idpf_map[] = {
>> + { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
>> + { .vendor_id = 0, /* sentinel */ },
>> +};
>> +
>> +struct idpf_adapter *
>> +idpf_find_adapter(struct rte_pci_device *pci_dev)
>> +{
>> + struct idpf_adapter *adapter;
>> +
>> + TAILQ_FOREACH(adapter, &adapter_list, next) {
>> + if (!strncmp(adapter->name, pci_dev->device.name,
>> PCI_PRI_STR_SIZE))
>> + return adapter;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static int
>> +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>> + struct rte_pci_device *pci_dev)
>> +{
>> + struct idpf_adapter *adapter;
>> + char name[RTE_ETH_NAME_MAX_LEN];
>> + int i, retval;
>> + bool first_probe = FALSE;
>> +
>> + if (!adapter_list_init) {
>> + TAILQ_INIT(&adapter_list);
>> + adapter_list_init = true;
>> + }
>> +
>> + adapter = idpf_find_adapter(pci_dev);
>> + if (!adapter) {
>> + first_probe = TRUE;
>> + adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
>> + sizeof(struct idpf_adapter), 0);
>> + if (!adapter) {
>> + PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
>> + return -1;
>> + }
>> +
>> + retval = idpf_adapter_init(pci_dev, adapter);
>> + if (retval) {
>> + PMD_INIT_LOG(ERR, "Failed to init adapter.");
>> + return retval;
>> + }
>> +
>> + TAILQ_INSERT_TAIL(&adapter_list, adapter, next);
>> + }
>> +
>> + retval = idpf_parse_devargs(pci_dev, adapter);
>> + if (retval) {
>> + PMD_INIT_LOG(ERR, "Failed to parse private devargs");
>> + goto err;
>> + }
>> +
>> + for (i = 0; i < adapter->req_vport_nb; i++) {
>> + snprintf(name, sizeof(name), "idpf_%s_vport_%d",
>> + pci_dev->device.name,
>> + adapter->req_vports[i]);
>> + retval = rte_eth_dev_create(&pci_dev->device, name,
>> + sizeof(struct idpf_vport),
>> + NULL, NULL, idpf_dev_init,
>> + adapter);
>> + if (retval)
>> + PMD_DRV_LOG(ERR, "failed to create vport %d",
>> + adapter->req_vports[i]);
>> + }
>> +
>> + return 0;
>> +
>> +err:
>> + if (first_probe) {
>> + TAILQ_REMOVE(&adapter_list, adapter, next);
>> + idpf_adapter_rel(adapter);
>> + rte_free(adapter);
>> + }
>> + return retval;
>> +}
>> +
>> +static void
>> +idpf_adapter_rel(struct idpf_adapter *adapter)
>> +{
>> + struct idpf_hw *hw = &adapter->hw;
>> + int i;
>> +
>> + idpf_ctlq_deinit(hw);
>> +
>> + rte_free(adapter->caps);
>> + adapter->caps = NULL;
>> +
>> + rte_free(adapter->mbx_resp);
>> + adapter->mbx_resp = NULL;
>> +
>> + if (adapter->vport_req_info) {
>> + for (i = 0; i < adapter->max_vport_nb; i++) {
>> + rte_free(adapter->vport_req_info[i]);
>> + adapter->vport_req_info[i] = NULL;
>> + }
>> + rte_free(adapter->vport_req_info);
>> + adapter->vport_req_info = NULL;
>> + }
>> +
>> + if (adapter->vport_recv_info) {
>> + for (i = 0; i < adapter->max_vport_nb; i++) {
>> + rte_free(adapter->vport_recv_info[i]);
>> + adapter->vport_recv_info[i] = NULL;
>> + }
>> + rte_free(adapter->vport_recv_info);
>> + adapter->vport_recv_info = NULL;
>> + }
>> +
>> + rte_free(adapter->vports);
>> + adapter->vports = NULL;
>> +}
>> +
>> +static int
>> +idpf_pci_remove(struct rte_pci_device *pci_dev)
>> +{
>> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
>> + uint16_t port_id;
>> +
>> + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through
>> rte_device */
>> + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
>> + rte_eth_dev_close(port_id);
>> + }
>> +
>> + TAILQ_REMOVE(&adapter_list, adapter, next);
>> + idpf_adapter_rel(adapter);
>> + rte_free(adapter);
>> +
>> + return 0;
>> +}
>> +
>> +static struct rte_pci_driver rte_idpf_pmd = {
>> + .id_table = pci_id_idpf_map,
>> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
>> + .probe = idpf_pci_probe,
>> + .remove = idpf_pci_remove,
>> +};
>> +
>> +/**
>> + * Driver initialization routine.
>> + * Invoked once at EAL init time.
>> + * Register itself as the [Poll Mode] Driver of PCI devices.
>> + */
>> +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
>> +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
>> +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic |
>> vfio-pci");
>> +
>> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
>> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
>> diff --git a/drivers/net/idpf/idpf_ethdev.h
>> b/drivers/net/idpf/idpf_ethdev.h
>> new file mode 100644
>> index 0000000000..c2d72fae11
>> --- /dev/null
>> +++ b/drivers/net/idpf/idpf_ethdev.h
>> @@ -0,0 +1,206 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2022 Intel Corporation
>> + */
>> +
>> +#ifndef _IDPF_ETHDEV_H_
>> +#define _IDPF_ETHDEV_H_
>> +
>> +#include <stdint.h>
>> +#include <rte_mbuf.h>
>> +#include <rte_mempool.h>
>> +#include <rte_malloc.h>
>> +#include <rte_spinlock.h>
>> +#include <rte_ethdev.h>
>> +#include <rte_kvargs.h>
>> +#include <ethdev_driver.h>
>> +#include <ethdev_pci.h>
>> +
>> +#include "idpf_logs.h"
>> +
>> +#include "idpf_osdep.h"
>> +#include "idpf_type.h"
>> +#include "idpf_devids.h"
>> +#include "idpf_lan_txrx.h"
>> +#include "idpf_lan_pf_regs.h"
>> +#include "virtchnl.h"
>> +#include "virtchnl2.h"
>> +
>> +#define IDPF_MAX_VPORT_NUM 8
>> +
>> +#define IDPF_DEFAULT_RXQ_NUM 16
>> +#define IDPF_DEFAULT_TXQ_NUM 16
>
> Two above defines are unused. It must be added when actually
> used. Please, double check all other defines below.
>
>> +
>> +#define IDPF_INVALID_VPORT_IDX 0xffff
>> +#define IDPF_TXQ_PER_GRP 1
>> +#define IDPF_TX_COMPLQ_PER_GRP 1
>> +#define IDPF_RXQ_PER_GRP 1
>> +#define IDPF_RX_BUFQ_PER_GRP 2
>> +
>> +#define IDPF_CTLQ_ID -1
>> +#define IDPF_CTLQ_LEN 64
>> +#define IDPF_DFLT_MBX_BUF_SIZE 4096
>> +
>> +#define IDPF_DFLT_Q_VEC_NUM 1
>> +#define IDPF_DFLT_INTERVAL 16
>> +
>> +#define IDPF_MAX_NUM_QUEUES 256
>> +#define IDPF_MIN_BUF_SIZE 1024
>> +#define IDPF_MAX_FRAME_SIZE 9728
>> +#define IDPF_MIN_FRAME_SIZE 14
>> +
>> +#define IDPF_NUM_MACADDR_MAX 64
>> +
>> +#define IDPF_MAX_PKT_TYPE 1024
>> +
>> +#define IDPF_VLAN_TAG_SIZE 4
>> +#define IDPF_ETH_OVERHEAD \
>> + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
>> +
>> +#ifndef ETH_ADDR_LEN
>> +#define ETH_ADDR_LEN 6
>> +#endif
>> +
>> +#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
>> +
>> +/* Message type read in virtual channel from PF */
>> +enum idpf_vc_result {
>> + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
>> + IDPF_MSG_NON, /* Read nothing from admin queue */
>> + IDPF_MSG_SYS, /* Read system msg from admin queue */
>> + IDPF_MSG_CMD, /* Read async command result */
>> +};
>> +
>> +struct idpf_chunks_info {
>> + uint32_t tx_start_qid;
>> + uint32_t rx_start_qid;
>> +
>> + uint64_t tx_qtail_start;
>> + uint32_t tx_qtail_spacing;
>> + uint64_t rx_qtail_start;
>> + uint32_t rx_qtail_spacing;
>> +};
>> +
>> +struct idpf_vport {
>> + struct idpf_adapter *adapter; /* Backreference to associated
>> adapter */
>> + uint16_t vport_id;
>> + uint16_t num_tx_q;
>> + uint16_t num_rx_q;
>> +
>> + uint16_t max_mtu;
>> + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
>> +
>> + uint16_t sw_idx; /* SW idx */
>> +
>> + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
>> + uint16_t max_pkt_len; /* Maximum packet length */
>> +
>> + /* Chunk info */
>> + struct idpf_chunks_info chunks_info;
>> +
>> + /* Event from ipf */
>> + bool link_up;
>> + uint32_t link_speed;
>> +
>> + uint16_t devarg_id;
>> + bool stopped;
>> + struct virtchnl2_vport_stats eth_stats_offset;
>> +};
>> +
>> +struct idpf_adapter {
>> + TAILQ_ENTRY(idpf_adapter) next;
>> + struct idpf_hw hw;
>> + char name[IDPF_ADAPTER_NAME_LEN];
>> +
>> + struct virtchnl_version_info virtchnl_version;
>> + struct virtchnl2_get_capabilities *caps;
>> +
>> + volatile enum virtchnl_ops pend_cmd; /* pending command not
>> finished */
>> + uint32_t cmd_retval; /* return value of the cmd response from ipf */
>> + uint8_t *mbx_resp; /* buffer to store the mailbox response from
>> ipf */
>> +
>> + /* Vport info */
>> + uint8_t **vport_req_info;
>> + uint8_t **vport_recv_info;
>> + struct idpf_vport **vports;
>> + uint16_t max_vport_nb;
>> + uint16_t req_vports[IDPF_MAX_VPORT_NUM];
>> + uint16_t req_vport_nb;
>> + uint16_t cur_vports;
>> + uint16_t cur_vport_nb;
>> + uint16_t cur_vport_idx;
>> +
>> + uint16_t used_vecs_num;
>> +
>> + /* Max config queue number per VC message */
>> + uint32_t max_rxq_per_msg;
>> + uint32_t max_txq_per_msg;
>> +
>> + uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
>> +
>> + bool stopped;
>> +};
>> +
>> +TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
>> +extern struct idpf_adapter_list adapter_list;
>> +
>> +#define IDPF_DEV_TO_PCI(eth_dev) \
>> + RTE_DEV_TO_PCI((eth_dev)->device)
>> +
>> +/* structure used for sending and checking response of virtchnl ops */
>> +struct idpf_cmd_info {
>> + uint32_t ops;
>> + uint8_t *in_args; /* buffer for sending */
>> + uint32_t in_args_size; /* buffer size for sending */
>> + uint8_t *out_buffer; /* buffer for response */
>> + uint32_t out_size; /* buffer size for response */
>> +};
>> +
>> +/* notify current command done. Only call in case execute
>> + * _atomic_set_cmd successfully.
>> + */
>> +static inline void
>> +_notify_cmd(struct idpf_adapter *adapter, int msg_ret)
>> +{
>> + adapter->cmd_retval = msg_ret;
>> + rte_wmb();
>
> Please, add a comment which explains why memory barier is
> requied here.
>
>> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
>> +}
>> +
>> +/* clear current command. Only call in case execute
>> + * _atomic_set_cmd successfully.
>> + */
>> +static inline void
>> +_clear_cmd(struct idpf_adapter *adapter)
>> +{
>> + rte_wmb();
>
> Please, add a comment which explains why memory barier is
> requied here.
>
>
>> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
>> + adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
>> +}
>> +
>> +/* Check there is pending cmd in execution. If none, set new command. */
>> +static inline int
>> +_atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
>
> It is bad to start a symbol from single underscore, since it is
> reservied for C compiler internal use.
>
>> +{
>> + enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
>> + int ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk,
>> &ops,
>> + 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
>> +
>> + if (!ret)
>> + PMD_DRV_LOG(ERR, "There is incomplete cmd %d",
>> adapter->pend_cmd);
>> +
>> + return !ret;
>> +}
>> +
>> +struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
>> +int idpf_dev_link_update(struct rte_eth_dev *dev,
>> + __rte_unused int wait_to_complete);
>> +void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
>> +int idpf_vc_check_api_version(struct idpf_adapter *adapter);
>> +int idpf_vc_get_caps(struct idpf_adapter *adapter);
>> +int idpf_vc_create_vport(struct rte_eth_dev *dev);
>> +int idpf_vc_destroy_vport(struct idpf_vport *vport);
>> +int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
>> +int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
>> + uint16_t buf_len, uint8_t *buf);
>> +
>> +#endif /* _IDPF_ETHDEV_H_ */
>> diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
>> new file mode 100644
>> index 0000000000..a64544f86e
>> --- /dev/null
>> +++ b/drivers/net/idpf/idpf_logs.h
>> @@ -0,0 +1,42 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2022 Intel Corporation
>> + */
>> +
>> +#ifndef _IDPF_LOGS_H_
>> +#define _IDPF_LOGS_H_
>> +
>> +#include <rte_log.h>
>> +
>> +extern int idpf_logtype_init;
>> +extern int idpf_logtype_driver;
>> +
>> +#define PMD_INIT_LOG(level, fmt, args...) \
>> + rte_log(RTE_LOG_ ## level, idpf_logtype_init, \
>> + "%s(): " fmt "\n", __func__, ##args)
>
> Please, use RTE_FMT/RTE_FMT_HEAD/RTE_FMT_TAIL if you
> want to extend format string. See examples in many
> drivers and libraries.
>
>> +
>> +#define PMD_INIT_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
>> +
>> +#define PMD_DRV_LOG_RAW(level, fmt, args...) \
>> + rte_log(RTE_LOG_ ## level, idpf_logtype_driver, \
>> + "%s(): " fmt "\n", __func__, ##args)
>> +
>> +#define PMD_DRV_LOG(level, fmt, args...) \
>> + PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
>> +
>> +#define PMD_DRV_FUNC_TRACE() PMD_DRV_LOG(DEBUG, " >>")
>> +
>> +#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
>> +#define PMD_RX_LOG(level, fmt, args...) \
>> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#else
>> +#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
>> +#endif
>> +
>> +#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
>> +#define PMD_TX_LOG(level, fmt, args...) \
>> + RTE_LOG(level, PMD, "%s(): " fmt "\n", __func__, ## args)
>> +#else
>> +#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
>> +#endif
>> +
>> +#endif /* _IDPF_LOGS_H_ */
>> diff --git a/drivers/net/idpf/idpf_vchnl.c
>> b/drivers/net/idpf/idpf_vchnl.c
>> new file mode 100644
>> index 0000000000..ef0288ff45
>> --- /dev/null
>> +++ b/drivers/net/idpf/idpf_vchnl.c
>> @@ -0,0 +1,495 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2022 Intel Corporation
>> + */
>> +
>> +#include <stdio.h>
>> +#include <errno.h>
>> +#include <stdint.h>
>> +#include <string.h>
>> +#include <unistd.h>
>> +#include <stdarg.h>
>> +#include <inttypes.h>
>> +#include <rte_byteorder.h>
>> +#include <rte_common.h>
>> +
>> +#include <rte_debug.h>
>> +#include <rte_atomic.h>
>> +#include <rte_eal.h>
>> +#include <rte_ether.h>
>> +#include <ethdev_driver.h>
>> +#include <ethdev_pci.h>
>> +#include <rte_dev.h>
>> +
>> +#include "idpf_ethdev.h"
>> +
>> +#include "idpf_prototype.h"
>> +
>> +#define IDPF_CTLQ_LEN 64
>> +
>> +static int
>> +idpf_vc_clean(struct idpf_adapter *adapter)
>> +{
>> + struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
>> + uint16_t num_q_msg = IDPF_CTLQ_LEN;
>> + struct idpf_dma_mem *dma_mem;
>> + int err = 0;
>> + uint32_t i;
>> +
>> + for (i = 0; i < 10; i++) {
>> + err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
>> + msleep(20);
>> + if (num_q_msg)
>> + break;
>> + }
>> + if (err)
>> + goto error;
>
> Why not just return err? Be consistent since you do not use
> goto to handle errors below.
>
>> +
>> + /* Empty queue is not an error */
>> + for (i = 0; i < num_q_msg; i++) {
>> + dma_mem = q_msg[i]->ctx.indirect.payload;
>> + if (dma_mem) {
>> + idpf_free_dma_mem(&adapter->hw, dma_mem);
>> + rte_free(dma_mem);
>> + }
>> + rte_free(q_msg[i]);
>> + }
>> +
>> +error:
>> + return err;
>> +}
>> +
>> +static int
>> +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
>> + uint16_t msg_size, uint8_t *msg)
>> +{
>> + struct idpf_ctlq_msg *ctlq_msg;
>> + struct idpf_dma_mem *dma_mem;
>> + int err = 0;
>> +
>> + err = idpf_vc_clean(adapter);
>> + if (err)
>> + goto err;
>> +
>> + ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
>> + sizeof(struct idpf_ctlq_msg), 0);
>> + if (!ctlq_msg) {
>> + err = -ENOMEM;
>> + goto err;
>> + }
>> +
>> + dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
>> + sizeof(struct idpf_dma_mem), 0);
>> + if (!dma_mem) {
>> + err = -ENOMEM;
>> + goto dma_mem_error;
>> + }
>> +
>> + dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
>> + idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
>> + if (!dma_mem->va) {
>> + err = -ENOMEM;
>> + goto dma_alloc_error;
>> + }
>> +
>> + memcpy(dma_mem->va, msg, msg_size);
>> +
>> + ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
>> + ctlq_msg->func_id = 0;
>> + ctlq_msg->data_len = msg_size;
>> + ctlq_msg->cookie.mbx.chnl_opcode = op;
>> + ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
>> + ctlq_msg->ctx.indirect.payload = dma_mem;
>> +
>> + err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
>> + if (err)
>> + goto send_error;
>> +
>> + return err;
>
> just return 0 to make it easier to read.
>
>> +
>> +send_error:
>> + idpf_free_dma_mem(&adapter->hw, dma_mem);
>> +dma_alloc_error:
>> + rte_free(dma_mem);
>> +dma_mem_error:
>> + rte_free(ctlq_msg);
>> +err:
>> + return err;
>> +}
>> +
>> +static enum idpf_vc_result
>> +idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
>> + uint8_t *buf)
>> +{
>> + struct idpf_hw *hw = &adapter->hw;
>> + struct idpf_ctlq_msg ctlq_msg;
>> + struct idpf_dma_mem *dma_mem = NULL;
>> + enum idpf_vc_result result = IDPF_MSG_NON;
>> + enum virtchnl_ops opcode;
>> + uint16_t pending = 1;
>> + int ret;
>> +
>> + ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
>> + if (ret) {
>> + PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
>> + if (ret != IDPF_ERR_CTLQ_NO_WORK)
>> + result = IDPF_MSG_ERR;
>> + return result;
>> + }
>> +
>> + rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
>> +
>> + opcode = (enum
>> virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
>> + adapter->cmd_retval =
>> + (enum
>> virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
>> +
>> + PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
>> + opcode, adapter->cmd_retval);
>> +
>> + if (opcode == VIRTCHNL2_OP_EVENT) {
>> + struct virtchnl2_event *ve =
>> + (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
>> +
>> + result = IDPF_MSG_SYS;
>> + switch (ve->event) {
>> + case VIRTCHNL2_EVENT_LINK_CHANGE:
>> + /* TBD */
>> + break;
>> + default:
>> + PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
>> + __func__, ve->event);
>> + break;
>> + }
>> + } else {
>> + /* async reply msg on command issued by pf previously */
>> + result = IDPF_MSG_CMD;
>> + if (opcode != adapter->pend_cmd) {
>> + PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
>> + adapter->pend_cmd, opcode);
>> + result = IDPF_MSG_ERR;
>> + }
>> + }
>> +
>> + if (ctlq_msg.data_len)
>> + dma_mem = ctlq_msg.ctx.indirect.payload;
>> + else
>> + pending = 0;
>> +
>> + ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
>> + if (ret && dma_mem)
>> + idpf_free_dma_mem(hw, dma_mem);
>> +
>> + return result;
>> +}
>> +
>> +#define MAX_TRY_TIMES 200
>> +#define ASQ_DELAY_MS 10
>> +
>> +int
>> +idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
>> uint16_t buf_len,
>> + uint8_t *buf)
>> +{
>> + int ret, err = 0, i = 0;
>
> It is very error prone to declare and initialize varialbes in
> the same line. Not a requirement, but just an advice to avoid
> it and have own line for each variable as you do below.
>
>> +
>> + do {
>> + ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
>> + if (ret == IDPF_MSG_CMD)
>> + break;
>> + rte_delay_ms(ASQ_DELAY_MS);
>> + } while (i++ < MAX_TRY_TIMES);
>> + if (i >= MAX_TRY_TIMES ||
>> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
>> + err = -1;
>> + PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd
>> %d",
>> + adapter->cmd_retval, ops);
>> + }
>> +
>> + return err;
>> +}
>> +
>> +static int
>> +idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct
>> idpf_cmd_info *args)
>> +{
>> + int err = 0;
>> + int i = 0;
>> + int ret;
>> +
>> + if (_atomic_set_cmd(adapter, args->ops))
>> + return -1;
>> +
>> + ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size,
>> args->in_args);
>> + if (ret) {
>> + PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
>> + _clear_cmd(adapter);
>> + return ret;
>> + }
>> +
>> + switch (args->ops) {
>> + case VIRTCHNL_OP_VERSION:
>> + case VIRTCHNL2_OP_GET_CAPS:
>> + case VIRTCHNL2_OP_CREATE_VPORT:
>> + case VIRTCHNL2_OP_DESTROY_VPORT:
>> + case VIRTCHNL2_OP_SET_RSS_KEY:
>> + case VIRTCHNL2_OP_SET_RSS_LUT:
>> + case VIRTCHNL2_OP_SET_RSS_HASH:
>> + case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
>> + case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
>> + case VIRTCHNL2_OP_ENABLE_QUEUES:
>> + case VIRTCHNL2_OP_DISABLE_QUEUES:
>> + case VIRTCHNL2_OP_ENABLE_VPORT:
>> + case VIRTCHNL2_OP_DISABLE_VPORT:
>> + case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
>> + case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
>> + case VIRTCHNL2_OP_ALLOC_VECTORS:
>> + case VIRTCHNL2_OP_DEALLOC_VECTORS:
>> + case VIRTCHNL2_OP_GET_STATS:
>> + /* for init virtchnl ops, need to poll the response */
>> + err = idpf_read_one_msg(adapter, args->ops, args->out_size,
>> args->out_buffer);
>> + _clear_cmd(adapter);
>> + break;
>> + case VIRTCHNL2_OP_GET_PTYPE_INFO:
>> + /* for multuple response message,
>> + * do not handle the response here.
>> + */
>> + break;
>> + default:
>> + /* For other virtchnl ops in running time,
>> + * wait for the cmd done flag.
>> + */
>> + do {
>> + if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
>> + break;
>> + rte_delay_ms(ASQ_DELAY_MS);
>> + /* If don't read msg or read sys event, continue */
>> + } while (i++ < MAX_TRY_TIMES);
>> + /* If there's no response is received, clear command */
>> + if (i >= MAX_TRY_TIMES ||
>> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
>> + err = -1;
>> + PMD_DRV_LOG(ERR, "No response or return failure (%d) for
>> cmd %d",
>> + adapter->cmd_retval, args->ops);
>> + _clear_cmd(adapter);
>> + }
>> + break;
>> + }
>> +
>> + return err;
>> +}
>> +
>> +int
>> +idpf_vc_check_api_version(struct idpf_adapter *adapter)
>> +{
>> + struct virtchnl_version_info version;
>> + struct idpf_cmd_info args;
>> + int err;
>> +
>> + memset(&version, 0, sizeof(struct virtchnl_version_info));
>> + version.major = VIRTCHNL_VERSION_MAJOR_2;
>> + version.minor = VIRTCHNL_VERSION_MINOR_0;
>> +
>> + args.ops = VIRTCHNL_OP_VERSION;
>> + args.in_args = (uint8_t *)&version;
>> + args.in_args_size = sizeof(version);
>> + args.out_buffer = adapter->mbx_resp;
>> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
>> +
>> + err = idpf_execute_vc_cmd(adapter, &args);
>> + if (err) {
>> + PMD_DRV_LOG(ERR,
>> + "Failed to execute command of VIRTCHNL_OP_VERSION");
>> + return err;
>> + }
>> +
>> + return err;
>
> Just return 0 here or no return err above.
>
>> +}
>> +
>> +int
>> +idpf_vc_get_caps(struct idpf_adapter *adapter)
>> +{
>> + struct virtchnl2_get_capabilities caps_msg;
>> + struct idpf_cmd_info args;
>> + int err;
>> +
>> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
>> + caps_msg.csum_caps =
>> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
>> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
>> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
>> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
>> +
>> + caps_msg.seg_caps =
>> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
>> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
>> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
>> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
>> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
>> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
>> + VIRTCHNL2_CAP_SEG_GENERIC;
>> +
>> + caps_msg.rss_caps =
>> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
>> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
>> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
>> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
>> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
>> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
>> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
>> + VIRTCHNL2_CAP_RSS_IPV4_AH |
>> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV6_AH |
>> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
>> +
>> + caps_msg.hsplit_caps =
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
>> +
>> + caps_msg.rsc_caps =
>> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
>> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
>> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
>> +
>> + caps_msg.other_caps =
>> + VIRTCHNL2_CAP_RDMA |
>> + VIRTCHNL2_CAP_SRIOV |
>> + VIRTCHNL2_CAP_MACFILTER |
>> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
>> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
>> + VIRTCHNL2_CAP_CRC |
>> + VIRTCHNL2_CAP_WB_ON_ITR |
>> + VIRTCHNL2_CAP_PROMISC |
>> + VIRTCHNL2_CAP_LINK_SPEED |
>> + VIRTCHNL2_CAP_VLAN;
>> +
>> + args.ops = VIRTCHNL2_OP_GET_CAPS;
>> + args.in_args = (uint8_t *)&caps_msg;
>> + args.in_args_size = sizeof(caps_msg);
>> + args.out_buffer = adapter->mbx_resp;
>> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
>> +
>> + err = idpf_execute_vc_cmd(adapter, &args);
>> + if (err) {
>> + PMD_DRV_LOG(ERR,
>> + "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
>> + return err;
>> + }
>> +
>> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
>> +
>> + return err;
>
> Just return 0 to make it easier to read.
>
>> +}
>> +
>> +int
>> +idpf_vc_create_vport(struct rte_eth_dev *dev)
>
> In general, the function does not look ethdev specific.
> If you have different classes of devices in the future,
> most likely you'll need the same code.
>
> Anyway I'd consider to make the prototype not ethdev
> specific and not PCI specific anyway. It looks like
> intput could be just 'struct idpf_adapter *'.
>
> It is better to keep all ethdev interface specifics in
> idpf_ethdev.c. Of course, it is not always possible, but
> it would make next-net maintainers life easier since it
> will limit scope of attentive review.
>
>> +{
>> + struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
>> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
>> + uint16_t idx = adapter->cur_vport_idx;
>> + struct virtchnl2_create_vport *vport_req_info =
>> + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
>> + struct virtchnl2_create_vport vport_msg;
>> + struct idpf_cmd_info args;
>> + int err = -1;
>> +
>> + memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
>> + vport_msg.vport_type = vport_req_info->vport_type;
>> + vport_msg.txq_model = vport_req_info->txq_model;
>> + vport_msg.rxq_model = vport_req_info->rxq_model;
>> + vport_msg.num_tx_q = vport_req_info->num_tx_q;
>> + vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
>> + vport_msg.num_rx_q = vport_req_info->num_rx_q;
>> + vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
>> +
>> + memset(&args, 0, sizeof(args));
>> + args.ops = VIRTCHNL2_OP_CREATE_VPORT;
>> + args.in_args = (uint8_t *)&vport_msg;
>> + args.in_args_size = sizeof(vport_msg);
>> + args.out_buffer = adapter->mbx_resp;
>> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
>> +
>> + err = idpf_execute_vc_cmd(adapter, &args);
>> + if (err) {
>> + PMD_DRV_LOG(ERR,
>> + "Failed to execute command of
>> VIRTCHNL2_OP_CREATE_VPORT");
>> + return err;
>> + }
>> +
>> + if (!adapter->vport_recv_info[idx]) {
>> + adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
>> + IDPF_DFLT_MBX_BUF_SIZE, 0);
>> + if (!adapter->vport_recv_info[idx]) {
>> + PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
>> + return err;
>
> You will return 0 here. It is better to be explicit and return
> just -1 instead of usage helper variable.
>
>> + }
>> + }
>> + rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
>> + IDPF_DFLT_MBX_BUF_SIZE);
>> + return err;
>
> just return 0.
>
>> +}
>> +
>> +int
>> +idpf_vc_destroy_vport(struct idpf_vport *vport)
>> +{
>> + struct idpf_adapter *adapter = vport->adapter;
>> + struct virtchnl2_vport vc_vport;
>> + struct idpf_cmd_info args;
>> + int err;
>> +
>> + vc_vport.vport_id = vport->vport_id;
>> +
>> + memset(&args, 0, sizeof(args));
>> + args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
>> + args.in_args = (uint8_t *)&vc_vport;
>> + args.in_args_size = sizeof(vc_vport);
>> + args.out_buffer = adapter->mbx_resp;
>> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
>> +
>> + err = idpf_execute_vc_cmd(adapter, &args);
>> + if (err) {
>> + PMD_DRV_LOG(ERR, "Failed to execute command of
>> VIRTCHNL2_OP_DESTROY_VPORT");
>> + return err;
>> + }
>> +
>> + return err;
>
> If you have return above, it would be easier to read
> if you return 0 here.
>
>> +}
>> +
>> +int
>> +idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
>> +{
>> + struct idpf_adapter *adapter = vport->adapter;
>> + struct virtchnl2_vport vc_vport;
>> + struct idpf_cmd_info args;
>> + int err;
>> +
>> + vc_vport.vport_id = vport->vport_id;
>> + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
>> + VIRTCHNL2_OP_DISABLE_VPORT;
>> + args.in_args = (u8 *)&vc_vport;
>> + args.in_args_size = sizeof(vc_vport);
>> + args.out_buffer = adapter->mbx_resp;
>> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
>> +
>> + err = idpf_execute_vc_cmd(adapter, &args);
>> + if (err) {
>> + PMD_DRV_LOG(ERR, "Failed to execute command of
>> VIRTCHNL2_OP_%s_VPORT",
>> + enable ? "ENABLE" : "DISABLE");
>> + }
>> +
>> + return err;
>
> Please, be consistent. You have return in if branch in the
> function above, but od not have return in this function
> whereas code pattern is absolutely the same.
>
>
>> +}
>> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
>> new file mode 100644
>> index 0000000000..bf8bf58ef5
>> --- /dev/null
>> +++ b/drivers/net/idpf/meson.build
>> @@ -0,0 +1,16 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2022 Intel Corporation
>> +
>> +if is_windows
>> + build = false
>> + reason = 'not supported on Windows'
>> + subdir_done()
>> +endif
>> +
>> +includes += include_directories('../../common/idpf')
>
> Isn't common dependency insufficient to have it in includes path?
>
>> +deps += ['common_idpf', 'security', 'cryptodev']
>> +
>> +sources = files(
>> + 'idpf_ethdev.c',
>> + 'idpf_vchnl.c',
>> +)
>
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 05/14] net/idpf: add support for queue start and stop
2022-10-21 5:18 ` [PATCH v9 05/14] net/idpf: add support for queue start and stop Junfeng Guo
@ 2022-10-21 7:53 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 7:53 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/21/22 08:18, Junfeng Guo wrote:
> Add support for queue operations:
> - rx_queue_start
> - rx_queue_stop
> - tx_queue_start
> - tx_queue_stop
Qeueues start/stop are required for deferred start.
So, before the patch device start must simply start
all setup queues which should reject deferred_start
request in queue configuration.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 06/14] net/idpf: add support for device information get
2022-10-21 5:18 ` [PATCH v9 06/14] net/idpf: add support for device information get Junfeng Guo
@ 2022-10-21 7:56 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 7:56 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev
On 10/21/22 08:18, Junfeng Guo wrote:
> Add dev ops dev_infos_get.
You can't test neither queues setup nor configure nor start
before the patch since ethdev layer uses
rte_eth_dev_info_get() and fails if it is not supported.
So, previous patches are not actually tested since patches
order is wrong.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 07/14] net/idpf: add support for packet type get
2022-10-21 5:18 ` [PATCH v9 07/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-21 8:00 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 8:00 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Wenjun Wu
On 10/21/22 08:18, Junfeng Guo wrote:
> Add dev ops dev_supported_ptypes_get.
It is nice, but I don't see code on datapath which
actually provides the packet type information.
Basically just after the patch you can't pretend
that it is actually supported since there is no
datapath yet. Also it does not allow me to check
the consistency - i.e. really check that datapath
supports all reported packet types.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-21 5:18 ` [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
@ 2022-10-21 8:29 ` Andrew Rybchenko
2022-10-24 13:26 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 8:29 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/21/22 08:18, Junfeng Guo wrote:
> Add Rx/Tx offloading support, including TSO and CHKSUM.
Typically code path for Rx and Tx offload are absolutely
different. So, I'm wondering why both are implemented in
one patch. If they are really independent, please, split
it into Rx and Tx patches to make it easier to review.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> doc/guides/nics/features/idpf.ini | 3 +
> drivers/net/idpf/idpf_ethdev.c | 10 ++
> drivers/net/idpf/idpf_rxtx.c | 212 +++++++++++++++++++++++++++++-
> drivers/net/idpf/idpf_rxtx.h | 19 +++
> 4 files changed, 242 insertions(+), 2 deletions(-)
>
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> index 79ffafc0d4..36dec39b9f 100644
> --- a/doc/guides/nics/features/idpf.ini
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -10,6 +10,9 @@
> Queue start/stop = Y
> Runtime Rx queue setup = Y
> Runtime Tx queue setup = Y
> +TSO = P
> +L3 checksum offload = P
> +L4 checksum offload = P
> Packet type parsing = Y
> Multiprocess aware = Y
> FreeBSD = Y
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index c6d6c87274..ffde2ce7d1 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -86,6 +86,16 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
> + dev_info->rx_offload_capa =
> + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
> +
> + dev_info->tx_offload_capa =
> + RTE_ETH_TX_OFFLOAD_TCP_TSO |
> + RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
I guess this one is supported from the very begining
implementation of your datapath. If so, it should be reported
in the patch which adds the datapath implementation.
> + RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is not mentined anywhereelse
in the patch. Is it actually supported? How?
>
> dev_info->default_rxconf = (struct rte_eth_rxconf) {
> .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> index 6280b7887b..31e266bbec 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -1269,6 +1269,47 @@ idpf_stop_queues(struct rte_eth_dev *dev)
> }
> }
>
> +#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
> + (BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
> +
> +static inline uint64_t
> +idpf_splitq_rx_csum_offload(uint8_t err)
> +{
> + uint64_t flags = 0;
> +
> + if (unlikely(!(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S))))
> + return flags;
> +
> + if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
> + flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
> + RTE_MBUF_F_RX_L4_CKSUM_GOOD);
> + return flags;
> + }
> +
> + if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)))
> + flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
> +
> + if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)))
> + flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
> +
> + if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)))
> + flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
> +
> + if (unlikely(err & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)))
> + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
> +
> + return flags;
> +}
> +
> static void
> idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
> {
> @@ -1343,9 +1384,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
> uint16_t pktlen_gen_bufq_id;
> struct idpf_rx_queue *rxq;
> const uint32_t *ptype_tbl;
> + uint8_t status_err0_qw1;
> struct rte_mbuf *rxm;
> uint16_t rx_id_bufq1;
> uint16_t rx_id_bufq2;
> + uint64_t pkt_flags;
> uint16_t pkt_len;
> uint16_t bufq_id;
> uint16_t gen_id;
> @@ -1420,6 +1463,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
> VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
> VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
>
> + status_err0_qw1 = rx_desc->status_err0_qw1;
> + pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
> +
> + rxm->ol_flags |= pkt_flags;
> +
> rx_pkts[nb_rx++] = rxm;
> }
>
> @@ -1496,6 +1544,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
> cq->tx_tail = next;
> }
>
> +/* Check if the context descriptor is needed for TX offloading */
> +static inline uint16_t
> +idpf_calc_context_desc(uint64_t flags)
> +{
> + if (flags & RTE_MBUF_F_TX_TCP_SEG)
> + return 1;
> +
> + return 0;
> +}
> +
> +/* set TSO context descriptor
> + */
> +static inline void
> +idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
> + union idpf_tx_offload tx_offload,
> + volatile union idpf_flex_tx_ctx_desc *ctx_desc)
> +{
> + uint16_t cmd_dtype;
> + uint32_t tso_len;
> + uint8_t hdr_len;
> +
> + if (!tx_offload.l4_len) {
> + PMD_TX_LOG(DEBUG, "L4 length set to 0");
> + return;
> + }
> +
> + hdr_len = tx_offload.l2_len +
> + tx_offload.l3_len +
> + tx_offload.l4_len;
> + cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
> + IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
> + tso_len = mbuf->pkt_len - hdr_len;
> +
> + ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
> + ctx_desc->tso.qw0.hdr_len = hdr_len;
> + ctx_desc->tso.qw0.mss_rt =
> + rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
> + IDPF_TXD_FLEX_CTX_MSS_RT_M);
> + ctx_desc->tso.qw0.flex_tlen =
> + rte_cpu_to_le_32(tso_len &
> + IDPF_TXD_FLEX_CTX_MSS_RT_M);
> +}
> +
> uint16_t
> idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts)
> @@ -1504,11 +1595,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> volatile struct idpf_flex_tx_sched_desc *txr;
> volatile struct idpf_flex_tx_sched_desc *txd;
> struct idpf_tx_entry *sw_ring;
> + union idpf_tx_offload tx_offload = {0};
> struct idpf_tx_entry *txe, *txn;
> uint16_t nb_used, tx_id, sw_id;
> struct rte_mbuf *tx_pkt;
> uint16_t nb_to_clean;
> uint16_t nb_tx = 0;
> + uint64_t ol_flags;
> + uint16_t nb_ctx;
>
> if (unlikely(!txq) || unlikely(!txq->q_started))
> return nb_tx;
> @@ -1538,8 +1632,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
>
> if (txq->nb_free < tx_pkt->nb_segs)
> break;
> - nb_used = tx_pkt->nb_segs;
>
> + ol_flags = tx_pkt->ol_flags;
> + tx_offload.l2_len = tx_pkt->l2_len;
> + tx_offload.l3_len = tx_pkt->l3_len;
> + tx_offload.l4_len = tx_pkt->l4_len;
> + tx_offload.tso_segsz = tx_pkt->tso_segsz;
> + /* Calculate the number of context descriptors needed. */
> + nb_ctx = idpf_calc_context_desc(ol_flags);
> + nb_used = tx_pkt->nb_segs + nb_ctx;
> +
> + /* context descriptor */
> + if (nb_ctx) {
> + volatile union idpf_flex_tx_ctx_desc *ctx_desc =
> + (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
> +
> + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
> + idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
> + ctx_desc);
> +
> + tx_id++;
> + if (tx_id == txq->nb_tx_desc)
> + tx_id = 0;
> + }
>
> do {
> txd = &txr[tx_id];
> @@ -1566,6 +1681,8 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
>
> if (unlikely(!(tx_id % 32)))
> txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
> + if (ol_flags & IDPF_TX_CKSUM_OFFLOAD_MASK)
> + txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_CS_EN;
> txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
> txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
> }
> @@ -1580,6 +1697,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> return nb_tx;
> }
>
> +#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
> + (BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
> + BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
> +
> +/* Translate the rx descriptor status and error fields to pkt flags */
> +static inline uint64_t
> +idpf_rxd_to_pkt_flags(uint16_t status_error)
> +{
> + uint64_t flags = 0;
> +
> + if (unlikely(!(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
> + return flags;
> +
> + if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
> + flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
Strictly speaking description of the
RTE_MBUF_F_RX_IP_CKSUM_GOOD says that IP checksum in the
patcket is valid, but there is no checksum in IPv6 header.
So, we can't say that it is valid for IPv6...
> + RTE_MBUF_F_RX_L4_CKSUM_GOOD);
> + return flags;
> + }
> +
> + if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)))
> + flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
> +
> + if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)))
> + flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
> +
> + if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)))
> + flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
> +
> + if (unlikely(status_error & BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)))
> + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
> + else
> + flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
> +
> + return flags;
> +}
> +
> static inline void
> idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
> uint16_t rx_id)
> @@ -1613,6 +1772,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
> struct rte_mbuf *rxm;
> struct rte_mbuf *nmb;
> uint16_t rx_status0;
> + uint64_t pkt_flags;
> uint64_t dma_addr;
> uint16_t nb_rx;
>
> @@ -1682,10 +1842,13 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
> rxm->data_len = rx_packet_len;
> rxm->port = rxq->port_id;
> rxm->ol_flags = 0;
> + pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
> rxm->packet_type =
> ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
> VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
>
> + rxm->ol_flags |= pkt_flags;
> +
>
> rx_pkts[nb_rx++] = rxm;
> }
> @@ -1748,14 +1911,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> {
> volatile struct idpf_flex_tx_desc *txd;
> volatile struct idpf_flex_tx_desc *txr;
> + union idpf_tx_offload tx_offload = {0};
> struct idpf_tx_entry *txe, *txn;
> struct idpf_tx_entry *sw_ring;
> struct idpf_tx_queue *txq;
> struct rte_mbuf *tx_pkt;
> struct rte_mbuf *m_seg;
> uint64_t buf_dma_addr;
> + uint64_t ol_flags;
> uint16_t tx_last;
> uint16_t nb_used;
> + uint16_t nb_ctx;
> uint16_t td_cmd;
> uint16_t tx_id;
> uint16_t nb_tx;
> @@ -1782,11 +1948,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> tx_pkt = *tx_pkts++;
> RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
>
> + ol_flags = tx_pkt->ol_flags;
> + tx_offload.l2_len = tx_pkt->l2_len;
> + tx_offload.l3_len = tx_pkt->l3_len;
> + tx_offload.l4_len = tx_pkt->l4_len;
> + tx_offload.tso_segsz = tx_pkt->tso_segsz;
> + /* Calculate the number of context descriptors needed. */
> + nb_ctx = idpf_calc_context_desc(ol_flags);
> +
> /* The number of descriptors that must be allocated for
> * a packet equals to the number of the segments of that
> * packet plus 1 context descriptor if needed.
> */
> - nb_used = (uint16_t)(tx_pkt->nb_segs);
> + nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
> tx_last = (uint16_t)(tx_id + nb_used - 1);
>
> /* Circular ring */
> @@ -1814,6 +1988,28 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
> }
> }
>
> + if (nb_ctx) {
> + /* Setup TX context descriptor if required */
> + volatile union idpf_flex_tx_ctx_desc *ctx_txd =
> + (volatile union idpf_flex_tx_ctx_desc *)
> + &txr[tx_id];
> +
> + txn = &sw_ring[txe->next_id];
> + RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
> + if (txe->mbuf) {
> + rte_pktmbuf_free_seg(txe->mbuf);
> + txe->mbuf = NULL;
> + }
> +
> + /* TSO enabled */
> + if (ol_flags & RTE_MBUF_F_TX_TCP_SEG)
> + idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
> + ctx_txd);
> +
> + txe->last_id = tx_last;
> + tx_id = txe->next_id;
> + txe = txn;
> + }
>
> m_seg = tx_pkt;
> do {
> @@ -1896,11 +2092,23 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
> return i;
> }
>
> + if (ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) {
> + rte_errno = ENOTSUP;
> + return i;
> + }
> +
> if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
> rte_errno = EINVAL;
> return i;
> }
>
> +#ifdef RTE_LIBRTE_ETHDEV_DEBUG
> + ret = rte_validate_tx_offload(m);
> + if (ret != 0) {
> + rte_errno = -ret;
> + return i;
> + }
> +#endif
> ret = rte_net_intel_cksum_prepare(m);
> if (ret != 0) {
> rte_errno = -ret;
> diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
> index bd3ebe2f50..a2394ffa2c 100644
> --- a/drivers/net/idpf/idpf_rxtx.h
> +++ b/drivers/net/idpf/idpf_rxtx.h
> @@ -44,6 +44,25 @@
> #define IDPF_MAX_TSO_FRAME_SIZE 262143
> #define IDPF_TX_MAX_MTU_SEG 10
>
> +#define IDPF_TX_CKSUM_OFFLOAD_MASK ( \
> + RTE_MBUF_F_TX_IP_CKSUM | \
> + RTE_MBUF_F_TX_L4_MASK | \
> + RTE_MBUF_F_TX_TCP_SEG)
> +
> +#define IDPF_TX_OFFLOAD_MASK ( \
> + RTE_MBUF_F_TX_OUTER_IPV6 | \
> + RTE_MBUF_F_TX_OUTER_IPV4 | \
> + RTE_MBUF_F_TX_IPV6 | \
> + RTE_MBUF_F_TX_IPV4 | \
> + RTE_MBUF_F_TX_VLAN | \
VLAN insertion offload is not supported yet.
> + RTE_MBUF_F_TX_IP_CKSUM | \
> + RTE_MBUF_F_TX_L4_MASK | \
> + RTE_MBUF_F_TX_TCP_SEG | \
> + RTE_ETH_TX_OFFLOAD_SECURITY)
What is RTE_ETH_TX_OFFLOAD_SECURITY doing in mbuf flags?
> +
> +#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
> + (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
> +
> #define IDPF_GET_PTYPE_SIZE(p) \
> (sizeof(struct virtchnl2_ptype) + \
> (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 10/14] net/idpf: add support for RSS
2022-10-21 5:18 ` [PATCH v9 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-21 8:38 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 8:38 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev
On 10/21/22 08:18, Junfeng Guo wrote:
> Add RSS support.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 123 ++++++++++++++++++++++++++++++++-
> drivers/net/idpf/idpf_ethdev.h | 26 +++++++
> drivers/net/idpf/idpf_rxtx.c | 27 ++++++++
> drivers/net/idpf/idpf_vchnl.c | 96 +++++++++++++++++++++++++
> 4 files changed, 269 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index ffde2ce7d1..cc1cfe402b 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -83,6 +83,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
> dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
>
> + dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
> dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
> @@ -90,7 +91,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
> RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
> - RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
> + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_RSS_HASH;
>
> dev_info->tx_offload_capa =
> RTE_ETH_TX_OFFLOAD_TCP_TSO |
> @@ -197,6 +199,10 @@ idpf_parse_devarg_id(char *name)
> return val;
> }
>
> +#ifndef IDPF_RSS_KEY_LEN
Why do we ned ifndef here?
> +#define IDPF_RSS_KEY_LEN 52
> +#endif
> +
> static int
> idpf_init_vport(struct rte_eth_dev *dev)
> {
> @@ -217,6 +223,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
> vport->max_mtu = vport_info->max_mtu;
> rte_memcpy(vport->default_mac_addr,
> vport_info->default_mac_addr, ETH_ALEN);
> + vport->rss_algorithm = vport_info->rss_algorithm;
> + vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
> + vport_info->rss_key_size);
> + vport->rss_lut_size = vport_info->rss_lut_size;
> vport->sw_idx = idx;
>
> for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> @@ -265,15 +275,116 @@ idpf_init_vport(struct rte_eth_dev *dev)
> }
>
> static int
> -idpf_dev_configure(__rte_unused struct rte_eth_dev *dev)
> +idpf_config_rss(struct idpf_vport *vport)
> +{
> + int ret;
> +
> + ret = idpf_vc_set_rss_key(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS key");
> + return ret;
> + }
> +
> + ret = idpf_vc_set_rss_lut(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
> + return ret;
> + }
> +
> + ret = idpf_vc_set_rss_hash(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
> + return ret;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_init_rss(struct idpf_vport *vport)
> +{
> + struct rte_eth_rss_conf *rss_conf;
> + uint16_t i, nb_q, lut_size;
> + int ret = 0;
> +
> + rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
> + nb_q = vport->dev_data->nb_rx_queues;
> +
> + vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
> + vport->rss_key_size, 0);
> + if (!vport->rss_key) {
> + PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
> + ret = -ENOMEM;
> + goto err_key;
> + }
> +
> + lut_size = vport->rss_lut_size;
> + vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
> + sizeof(uint32_t) * lut_size, 0);
> + if (!vport->rss_lut) {
> + PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
> + ret = -ENOMEM;
> + goto err_lut;
> + }
> +
> + if (!rss_conf->rss_key) {
> + for (i = 0; i < vport->rss_key_size; i++)
> + vport->rss_key[i] = (uint8_t)rte_rand();
> + } else {
> + rte_memcpy(vport->rss_key, rss_conf->rss_key,
> + RTE_MIN(rss_conf->rss_key_len,
> + vport->rss_key_size));
If provides length is smaller than required, shouldn't we
fill in the rest using rte_rand() as above?
Since the RSS key length is reported in device info,
IMHO it is better to require from user to provide RSS
key with correct legnth.
> + }
> +
> + for (i = 0; i < lut_size; i++)
> + vport->rss_lut[i] = i % nb_q;
Are you sure that nb_q is not 0? In theory it is possible
to configure and start with Tx queues only.
> +
> + vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
> +
> + ret = idpf_config_rss(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to configure RSS");
> + goto err_cfg;
> + }
> +
> + return ret;
> +
> +err_cfg:
> + rte_free(vport->rss_lut);
> + vport->rss_lut = NULL;
> +err_lut:
> + rte_free(vport->rss_key);
> + vport->rss_key = NULL;
> +err_key:
> + return ret;
> +}
> +
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev)
> {
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + int ret = 0;
> +
> if (dev->data->nb_tx_queues > IDPF_DEFAULT_TXQ_NUM ||
> dev->data->nb_rx_queues > IDPF_DEFAULT_RXQ_NUM) {
> PMD_INIT_LOG(ERR, "Invalid queue number.");
> return -EINVAL;
> }
>
> - return 0;
> + if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
> + dev->data->dev_conf.rxmode.offloads |=
> + RTE_ETH_RX_OFFLOAD_RSS_HASH;
Why? User controls offloads.
> +
> + if (adapter->caps->rss_caps) {
> + ret = idpf_init_rss(vport);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init rss");
> + return ret;
> + }
If RSS is not supported but requested, it must be an error, not
simply ignoring of the RSS request.
> + }
> +
> + return ret;
> }
>
> static int
> @@ -372,6 +483,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
> idpf_dev_stop(dev);
> idpf_vc_destroy_vport(vport);
>
> + rte_free(vport->rss_lut);
> + vport->rss_lut = NULL;
> +
> + rte_free(vport->rss_key);
> + vport->rss_key = NULL;
> +
> adapter->cur_vports &= ~BIT(vport->devarg_id);
>
> rte_free(vport);
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> index d414efc917..c25fa2e3ce 100644
> --- a/drivers/net/idpf/idpf_ethdev.h
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -56,6 +56,20 @@
> #define IDPF_ETH_OVERHEAD \
> (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
>
> +#define IDPF_RSS_OFFLOAD_ALL ( \
> + RTE_ETH_RSS_IPV4 | \
> + RTE_ETH_RSS_FRAG_IPV4 | \
> + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
> + RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
> + RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
> + RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
> + RTE_ETH_RSS_IPV6 | \
> + RTE_ETH_RSS_FRAG_IPV6 | \
> + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
> + RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
> + RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
> + RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
> +
> #ifndef ETH_ADDR_LEN
> #define ETH_ADDR_LEN 6
> #endif
> @@ -102,11 +116,20 @@ struct idpf_vport {
> uint16_t max_mtu;
> uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
>
> + enum virtchnl_rss_algorithm rss_algorithm;
> + uint16_t rss_key_size;
> + uint16_t rss_lut_size;
> +
> uint16_t sw_idx; /* SW idx */
>
> struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
> uint16_t max_pkt_len; /* Maximum packet length */
>
> + /* RSS info */
> + uint32_t *rss_lut;
> + uint8_t *rss_key;
> + uint64_t rss_hf;
> +
> /* Chunk info */
> struct idpf_chunks_info chunks_info;
>
> @@ -216,6 +239,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
> int idpf_vc_get_caps(struct idpf_adapter *adapter);
> int idpf_vc_create_vport(struct rte_eth_dev *dev);
> int idpf_vc_destroy_vport(struct idpf_vport *vport);
> +int idpf_vc_set_rss_key(struct idpf_vport *vport);
> +int idpf_vc_set_rss_lut(struct idpf_vport *vport);
> +int idpf_vc_set_rss_hash(struct idpf_vport *vport);
> int idpf_vc_config_rxqs(struct idpf_vport *vport);
> int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
> int idpf_vc_config_txqs(struct idpf_vport *vport);
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> index 31e266bbec..05976acf7f 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -1310,6 +1310,32 @@ idpf_splitq_rx_csum_offload(uint8_t err)
> return flags;
> }
>
> +#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
> +#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
> +#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
> +
> +static inline uint64_t
> +idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
> + volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
> +{
> + uint8_t status_err0_qw0;
> + uint64_t flags = 0;
> +
> + status_err0_qw0 = rx_desc->status_err0_qw0;
> +
> + if (status_err0_qw0 & BIT(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) {
> + flags |= RTE_MBUF_F_RX_RSS_HASH;
> + mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
> + IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
> + ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
> + IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
> + ((uint32_t)(rx_desc->hash3) <<
> + IDPF_RX_FLEX_DESC_ADV_HASH3_S);
> + }
> +
> + return flags;
> +}
> +
> static void
> idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
> {
> @@ -1465,6 +1491,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
>
> status_err0_qw1 = rx_desc->status_err0_qw1;
> pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
> + pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
>
> rxm->ol_flags |= pkt_flags;
>
> diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
> index 7389128712..8f447046b1 100644
> --- a/drivers/net/idpf/idpf_vchnl.c
> +++ b/drivers/net/idpf/idpf_vchnl.c
> @@ -679,6 +679,102 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
> return err;
> }
>
> +int
> +idpf_vc_set_rss_key(struct idpf_vport *vport)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_rss_key *rss_key;
> + struct idpf_cmd_info args;
> + int len, err;
> +
> + len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
> + (vport->rss_key_size - 1);
> + rss_key = rte_zmalloc("rss_key", len, 0);
> + if (!rss_key)
> + return -ENOMEM;
> +
> + rss_key->vport_id = vport->vport_id;
> + rss_key->key_len = vport->rss_key_size;
> + rte_memcpy(rss_key->key, vport->rss_key,
> + sizeof(rss_key->key[0]) * vport->rss_key_size);
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
> + args.in_args = (uint8_t *)rss_key;
> + args.in_args_size = len;
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
> + return err;
> + }
> +
> + rte_free(rss_key);
> + return err;
> +}
> +
> +int
> +idpf_vc_set_rss_lut(struct idpf_vport *vport)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_rss_lut *rss_lut;
> + struct idpf_cmd_info args;
> + int len, err;
> +
> + len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
> + (vport->rss_lut_size - 1);
> + rss_lut = rte_zmalloc("rss_lut", len, 0);
> + if (!rss_lut)
> + return -ENOMEM;
> +
> + rss_lut->vport_id = vport->vport_id;
> + rss_lut->lut_entries = vport->rss_lut_size;
> + rte_memcpy(rss_lut->lut, vport->rss_lut,
> + sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
> + args.in_args = (uint8_t *)rss_lut;
> + args.in_args_size = len;
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err)
> + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
> +
> + rte_free(rss_lut);
> + return err;
> +}
> +
> +int
> +idpf_vc_set_rss_hash(struct idpf_vport *vport)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_rss_hash rss_hash;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&rss_hash, 0, sizeof(rss_hash));
> + rss_hash.ptype_groups = vport->rss_hf;
> + rss_hash.vport_id = vport->vport_id;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
> + args.in_args = (uint8_t *)&rss_hash;
> + args.in_args_size = sizeof(rss_hash);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err)
> + PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
> +
> + return err;
> +}
> +
> #define IDPF_RX_BUF_STRIDE 64
> int
> idpf_vc_config_rxqs(struct idpf_vport *vport)
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v9 01/14] common/idpf: introduce common library
2022-10-21 6:40 ` Andrew Rybchenko
@ 2022-10-21 12:35 ` Xing, Beilei
2022-10-21 12:38 ` Andrew Rybchenko
2022-10-21 12:46 ` Zhang, Qi Z
0 siblings, 2 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-21 12:35 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, October 21, 2022 2:40 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v9 01/14] common/idpf: introduce common library
>
> On 10/21/22 08:18, Junfeng Guo wrote:
> > Introduce common library for IDPF (Infrastructure Data Path Function)
> > PMD.
> >
> > Also add OS specific implementation about some MACRO definitions and
> > small functions which are specific for DPDK.
>
> Common drivers are required when different class drivers need to share
> some code. So, it must be expalined here why do you create common driver
> instead of usage of base/ driver in your net driver.
Hi Andrew,
Thanks for all your comments.
The common driver will be also used in another PMD which should be upstream in 23.03, so we create it.
>
> Note that common driver is a DPDK driver and it must follow DPDK coding
> style. If the code is actually shared with something else and do not follow
> DPDK coding style from the very beginning (since it is an existing code), it
> should be in base/ subdir in either common driver or net driver.
The common driver is a BSD license release provided by Intel internal team, basically we won't change it. It's the same process as Intel other PMDs, such as base driver of iavf, etc.
Is it OK if it's in common/idpf/base/ folder?
>
> Also you should not use own trivial wrappers for DPDK API in DPDK-specific
> code. It just complicates reading.
> E.g. BIT() vs RTE_BIT32().
Make sense, will try my best to address all your comments in the next version.
Thanks again.
>
> So, I need an answer on above questions before I continue review.
>
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > diff --git a/drivers/common/idpf/idpf_alloc.h
> > b/drivers/common/idpf/idpf_alloc.h
> > new file mode 100644
> > index 0000000000..bc054851b3
> > --- /dev/null
> > +++ b/drivers/common/idpf/idpf_alloc.h
> > @@ -0,0 +1,22 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2001-2022 Intel Corporation */
> > +
> > +#ifndef _IDPF_ALLOC_H_
> > +#define _IDPF_ALLOC_H_
> > +
> > +/* Memory types */
>
> If it is a DPDK-specific driver and it is an interface provided by common driver,
> it should use Doxygen-style comments to be a part of genereated API
> documentation.
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 01/14] common/idpf: introduce common library
2022-10-21 12:35 ` Xing, Beilei
@ 2022-10-21 12:38 ` Andrew Rybchenko
2022-10-21 12:46 ` Zhang, Qi Z
1 sibling, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-21 12:38 UTC (permalink / raw)
To: Xing, Beilei, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing; +Cc: dev, Wang, Xiao W
On 10/21/22 15:35, Xing, Beilei wrote:
>
>
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Friday, October 21, 2022 2:40 PM
>> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
>> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
>> <beilei.xing@intel.com>
>> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
>> Subject: Re: [PATCH v9 01/14] common/idpf: introduce common library
>>
>> On 10/21/22 08:18, Junfeng Guo wrote:
>>> Introduce common library for IDPF (Infrastructure Data Path Function)
>>> PMD.
>>>
>>> Also add OS specific implementation about some MACRO definitions and
>>> small functions which are specific for DPDK.
>>
>> Common drivers are required when different class drivers need to share
>> some code. So, it must be expalined here why do you create common driver
>> instead of usage of base/ driver in your net driver.
>
> Hi Andrew,
> Thanks for all your comments.
> The common driver will be also used in another PMD which should be upstream in 23.03, so we create it.
>
>>
>> Note that common driver is a DPDK driver and it must follow DPDK coding
>> style. If the code is actually shared with something else and do not follow
>> DPDK coding style from the very beginning (since it is an existing code), it
>> should be in base/ subdir in either common driver or net driver.
>
> The common driver is a BSD license release provided by Intel internal team, basically we won't change it. It's the same process as Intel other PMDs, such as base driver of iavf, etc.
> Is it OK if it's in common/idpf/base/ folder?
If so, yes, it should be there. Some common DPDK-specifics bits
could be in common/idpf, but code shared with other teams
should be in base/.
>
>>
>> Also you should not use own trivial wrappers for DPDK API in DPDK-specific
>> code. It just complicates reading.
>> E.g. BIT() vs RTE_BIT32().
>
> Make sense, will try my best to address all your comments in the next version.
> Thanks again.
>
>>
>> So, I need an answer on above questions before I continue review.
>>
>>>
>>> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
>>> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
>>> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>>
>> [snip]
>>
>>> diff --git a/drivers/common/idpf/idpf_alloc.h
>>> b/drivers/common/idpf/idpf_alloc.h
>>> new file mode 100644
>>> index 0000000000..bc054851b3
>>> --- /dev/null
>>> +++ b/drivers/common/idpf/idpf_alloc.h
>>> @@ -0,0 +1,22 @@
>>> +/* SPDX-License-Identifier: BSD-3-Clause
>>> + * Copyright(c) 2001-2022 Intel Corporation */
>>> +
>>> +#ifndef _IDPF_ALLOC_H_
>>> +#define _IDPF_ALLOC_H_
>>> +
>>> +/* Memory types */
>>
>> If it is a DPDK-specific driver and it is an interface provided by common driver,
>> it should use Doxygen-style comments to be a part of genereated API
>> documentation.
>
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v9 02/14] net/idpf: add support for device initialization
2022-10-21 7:48 ` Andrew Rybchenko
@ 2022-10-21 12:41 ` Zhang, Qi Z
2022-10-25 7:52 ` Andrew Rybchenko
0 siblings, 1 reply; 376+ messages in thread
From: Zhang, Qi Z @ 2022-10-21 12:41 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Wu, Jingjing, Xing, Beilei
Cc: dev, Li, Xiaoyun, Wang, Xiao W
Hi Andrew:
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, October 21, 2022 3:48 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v9 02/14] net/idpf: add support for device initialization
>
> On 10/21/22 10:39, Andrew Rybchenko wrote:
> > On 10/21/22 08:18, Junfeng Guo wrote:
> >> Support device init and add the following dev ops skeleton:
> >> - dev_configure
> >> - dev_start
>
> One more question: are you sure that you can start without
> queues setup?
I agree the patch set is not well-ordered. Keeping each increment be measurable is a good practice. We should keep this in mind.
But not sure if this is a blocking issue for patch merging as we saw the risk to have this in RC2 if we must to rework for this.
Btw, the first version of idpf PMD is quite simple. It's very similar to iavf PMD but has a few features, so I hope the out-of-order issue may not downgrade the readability :)
Please kindly give your suggestion.
Thanks
Qi
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v9 01/14] common/idpf: introduce common library
2022-10-21 12:35 ` Xing, Beilei
2022-10-21 12:38 ` Andrew Rybchenko
@ 2022-10-21 12:46 ` Zhang, Qi Z
1 sibling, 0 replies; 376+ messages in thread
From: Zhang, Qi Z @ 2022-10-21 12:46 UTC (permalink / raw)
To: Xing, Beilei, Andrew Rybchenko, Guo, Junfeng, Wu, Jingjing
Cc: dev, Wang, Xiao W
> -----Original Message-----
> From: Xing, Beilei <beilei.xing@intel.com>
> Sent: Friday, October 21, 2022 8:35 PM
> To: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Guo, Junfeng
> <junfeng.guo@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: RE: [PATCH v9 01/14] common/idpf: introduce common library
>
>
>
> > -----Original Message-----
> > From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> > Sent: Friday, October 21, 2022 2:40 PM
> > To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> > <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> > Beilei <beilei.xing@intel.com>
> > Cc: dev@dpdk.org; Wang, Xiao W <xiao.w.wang@intel.com>
> > Subject: Re: [PATCH v9 01/14] common/idpf: introduce common library
> >
> > On 10/21/22 08:18, Junfeng Guo wrote:
> > > Introduce common library for IDPF (Infrastructure Data Path
> > > Function) PMD.
> > >
> > > Also add OS specific implementation about some MACRO definitions and
> > > small functions which are specific for DPDK.
> >
> > Common drivers are required when different class drivers need to share
> > some code. So, it must be expalined here why do you create common
> > driver instead of usage of base/ driver in your net driver.
>
> Hi Andrew,
> Thanks for all your comments.
> The common driver will be also used in another PMD which should be
> upstream in 23.03, so we create it.
>
> >
> > Note that common driver is a DPDK driver and it must follow DPDK
> > coding style. If the code is actually shared with something else and
> > do not follow DPDK coding style from the very beginning (since it is
> > an existing code), it should be in base/ subdir in either common driver or
> net driver.
>
> The common driver is a BSD license release provided by Intel internal team,
> basically we won't change it. It's the same process as Intel other PMDs, such
> as base driver of iavf, etc.
> Is it OK if it's in common/idpf/base/ folder?
+1 , we'd better move the code only shared by DPDK PMD and kernel driver into the base folder, and we add code only shared by different DPDK PMDs in the upper folder in future.
>
> >
> > Also you should not use own trivial wrappers for DPDK API in
> > DPDK-specific code. It just complicates reading.
> > E.g. BIT() vs RTE_BIT32().
>
> Make sense, will try my best to address all your comments in the next version.
> Thanks again.
>
> >
> > So, I need an answer on above questions before I continue review.
> >
> > >
> > > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> >
> > [snip]
> >
> > > diff --git a/drivers/common/idpf/idpf_alloc.h
> > > b/drivers/common/idpf/idpf_alloc.h
> > > new file mode 100644
> > > index 0000000000..bc054851b3
> > > --- /dev/null
> > > +++ b/drivers/common/idpf/idpf_alloc.h
> > > @@ -0,0 +1,22 @@
> > > +/* SPDX-License-Identifier: BSD-3-Clause
> > > + * Copyright(c) 2001-2022 Intel Corporation */
> > > +
> > > +#ifndef _IDPF_ALLOC_H_
> > > +#define _IDPF_ALLOC_H_
> > > +
> > > +/* Memory types */
> >
> > If it is a DPDK-specific driver and it is an interface provided by
> > common driver, it should use Doxygen-style comments to be a part of
> > genereated API documentation.
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 00/14] add support for idpf PMD in DPDK
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-21 6:40 ` Andrew Rybchenko
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 01/14] net/idpf: add support for device start and stop Junfeng Guo
` (13 more replies)
1 sibling, 14 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v10:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
Junfeng Guo (14):
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for packet type get
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
doc/guides/nics/features/idpf.ini | 7 +
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 376 +++++-
drivers/net/idpf/idpf_ethdev.h | 66 +
drivers/net/idpf/idpf_rxtx.c | 1555 +++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 172 +++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 ++
drivers/net/idpf/idpf_vchnl.c | 964 ++++++++++++++
drivers/net/idpf/meson.build | 28 +
10 files changed, 4155 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 01/14] net/idpf: add support for device start and stop
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 02/14] net/idpf: add support for queue start Junfeng Guo
` (12 subsequent siblings)
13 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 89 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 5 ++
2 files changed, 94 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1d2075f466..4c7a2d0748 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,17 +29,42 @@ static const char * const idpf_valid_args[] = {
};
static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
@@ -233,6 +258,70 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+
+ PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
+ return -ENOTSUP;
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+
+ PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
+ return -ENOTSUP;
+ }
+
+ return err;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -1;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ if (idpf_start_queues(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return -1;
+ }
+
+ if (idpf_vc_ena_dis_vport(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c0ae801fd5..070531cc48 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -105,6 +105,9 @@ struct idpf_vport {
/* Chunk info */
struct idpf_chunks_info chunks_info;
+ /* Event from ipf */
+ bool link_up;
+
uint16_t devarg_id;
};
@@ -195,6 +198,8 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
}
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 02/14] net/idpf: add support for queue start
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 01/14] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 03/14] net/idpf: add support for queue stop Junfeng Guo
` (11 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 18 +-
drivers/net/idpf/idpf_ethdev.h | 7 +
drivers/net/idpf/idpf_rxtx.c | 214 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 4 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 684 insertions(+), 6 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4c7a2d0748..954bd0bf4b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -270,18 +272,22 @@ idpf_start_queues(struct rte_eth_dev *dev)
txq = dev->data->tx_queues[i];
if (txq == NULL || txq->tx_deferred_start)
continue;
-
- PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
- return -ENOTSUP;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
}
for (i = 0; i < dev->data->nb_rx_queues; i++) {
rxq = dev->data->rx_queues[i];
if (rxq == NULL || rxq->rx_deferred_start)
continue;
-
- PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
- return -ENOTSUP;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
}
return err;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 070531cc48..094f123be1 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -205,6 +205,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 76669504c3..b71c1ac6db 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -796,3 +796,217 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a79e9a0b26..d076903e3c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -115,8 +115,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ec72a8bba7..5e5508e999 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_CTLQ_LEN 64
@@ -227,6 +228,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -444,6 +449,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 03/14] net/idpf: add support for queue stop
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 01/14] net/idpf: add support for device start and stop Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 02/14] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 04/14] net/idpf: add queue release Junfeng Guo
` (10 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++-
drivers/net/idpf/idpf_rxtx.c | 140 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 11 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
5 files changed, 232 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 7a44b8b5e4..30e1c0831e 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,4 +7,5 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 954bd0bf4b..6040050dd9 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -62,7 +62,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -300,22 +302,26 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -1;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return -1;
+ goto err_mtu;
}
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- return -1;
+ goto err_vport;
}
return 0;
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return -1;
}
static int
@@ -325,6 +331,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
+ idpf_stop_queues(dev);
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index b71c1ac6db..c14fd4324e 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static inline void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -311,6 +360,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -552,6 +602,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -654,6 +705,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -778,6 +830,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -1010,3 +1063,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index d076903e3c..e1de436a4a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -111,16 +111,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 5e5508e999..9138799989 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -891,6 +891,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 04/14] net/idpf: add queue release
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 03/14] net/idpf: add support for queue stop Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 05/14] net/idpf: add support for packet type get Junfeng Guo
` (9 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6040050dd9..4c058660b4 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -66,7 +66,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index c14fd4324e..298eaf0a1a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -120,6 +120,51 @@ static const struct idpf_txq_ops def_txq_ops = {
.release_mbufs = release_txq_mbufs,
};
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -401,6 +446,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -541,6 +592,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -654,6 +711,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -776,6 +839,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1124,6 +1193,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index e1de436a4a..9f767079b2 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -126,12 +126,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 05/14] net/idpf: add support for packet type get
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 04/14] net/idpf: add queue release Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 06/14] net/idpf: add support for MTU configuration Junfeng Guo
` (8 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_vchnl.c | 240 ++++++++++++++++++++++++++++++
6 files changed, 280 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 30e1c0831e..a03068df85 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,4 +8,5 @@
;
[Features]
Queue start/stop = Y
+Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4c058660b4..80826569c8 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -641,6 +642,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 094f123be1..58f070c976 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -47,6 +47,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -141,6 +143,8 @@ struct idpf_adapter {
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
bool stopped;
};
@@ -202,6 +206,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -213,6 +218,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 298eaf0a1a..c3c4acb69f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 9f767079b2..11f1fde5d4 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,10 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,4 +141,7 @@ int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9138799989..86a55d2ff1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -238,6 +238,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -286,6 +291,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -984,3 +1198,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 06/14] net/idpf: add support for MTU configuration
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 05/14] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 07/14] net/idpf: add support for basic Rx datapath Junfeng Guo
` (7 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 15 +++++++++++++++
2 files changed, 16 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index a03068df85..d4eb9b374c 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,5 +8,6 @@
;
[Features]
Queue start/stop = Y
+MTU update = Y
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 80826569c8..1654af0053 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -34,6 +34,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -72,6 +73,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -122,6 +124,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -209,6 +223,7 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
vport->sw_idx = idx;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 07/14] net/idpf: add support for basic Rx datapath
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 06/14] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 08/14] net/idpf: add support for basic Tx datapath Junfeng Guo
` (6 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 286 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 5 +
3 files changed, 293 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1654af0053..f03df0a3ef 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -330,6 +330,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index c3c4acb69f..4037cf5a74 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1250,3 +1250,289 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 11f1fde5d4..056596ce94 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -140,8 +140,13 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 08/14] net/idpf: add support for basic Tx datapath
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 07/14] net/idpf: add support for basic Rx datapath Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 09/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (5 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 353 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 8 +
4 files changed, 364 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f03df0a3ef..d3ac269419 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -94,7 +94,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
dev_info->rx_offload_capa = 0;
- dev_info->tx_offload_capa = 0;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
@@ -331,6 +331,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 58f070c976..238c641c3f 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -44,6 +44,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4037cf5a74..fd203c985d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1415,6 +1415,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1526,6 +1668,204 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1536,3 +1876,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 056596ce94..eb0b230d3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -144,9 +144,17 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 09/14] net/idpf: add support for write back based on ITR expire
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 08/14] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 10/14] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 111 +++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d3ac269419..57e4bf0d67 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -280,6 +280,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps != 0) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -317,6 +401,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
@@ -325,6 +413,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -354,6 +459,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -365,6 +474,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -717,6 +831,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 238c641c3f..d89a06d239 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -42,6 +42,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -105,6 +108,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -140,6 +148,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -219,6 +229,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 86a55d2ff1..34369cfc96 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -1024,6 +1028,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 10/14] net/idpf: add support for RSS
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 09/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 11/14] net/idpf: add support for Rx offloading Junfeng Guo
` (3 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 ++++++++
drivers/net/idpf/idpf_vchnl.c | 97 +++++++++++++++++++++++++++
3 files changed, 237 insertions(+), 2 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 57e4bf0d67..739cf31d65 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,6 +90,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -226,6 +227,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -271,13 +276,114 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
- if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0)
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0) {
PMD_INIT_LOG(ERR, "RSS is not supported.");
+ }
- return 0;
+ return ret;
}
static int
@@ -474,6 +580,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index d89a06d239..7fe2647956 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -57,6 +57,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -103,11 +117,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -221,6 +244,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 34369cfc96..63605f2f64 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -228,6 +228,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -667,6 +670,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 11/14] net/idpf: add support for Rx offloading
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 10/14] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 12/14] net/idpf: add support for Tx offloading Junfeng Guo
` (2 subsequent siblings)
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
drivers/net/idpf/idpf_ethdev.c | 9 ++-
drivers/net/idpf/idpf_rxtx.c | 122 ++++++++++++++++++++++++++++++
3 files changed, 132 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d4eb9b374c..c86d9378ea 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,5 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 739cf31d65..d8cc423a23 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -94,7 +94,14 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
- dev_info->rx_offload_capa = 0;
+
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index fd203c985d..143c8b69f3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1250,6 +1250,72 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
@@ -1325,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1393,11 +1461,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1557,6 +1632,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1590,6 +1707,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1655,10 +1773,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 12/14] net/idpf: add support for Tx offloading
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 11/14] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Tx offloading support:
- support TSO
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 134 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 158 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c86d9378ea..47c686762d 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,6 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Packet type parsing = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d8cc423a23..21315866bf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -102,7 +102,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_RSS_HASH;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 143c8b69f3..8f82cf1b59 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1549,6 +1549,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1557,11 +1600,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1591,7 +1637,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1842,14 +1910,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1876,11 +1947,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1908,6 +1987,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1968,16 +2070,44 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
int i, ret;
+ uint64_t ol_flags;
struct rte_mbuf *m;
for (i = 0; i < nb_pkts; i++) {
m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index eb0b230d3a..efb2734d85 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,16 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 13/14] net/idpf: add AVX512 data path for single queue model
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 12/14] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 14/14] net/idpf: add support for timestamp offload Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 22 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1190 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7fe2647956..2485b3a784 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -180,6 +180,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8f82cf1b59..abef84b3b0 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -271,6 +273,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static inline void
@@ -2118,25 +2122,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index efb2734d85..e808710b41 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -52,6 +58,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -82,6 +93,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -166,12 +181,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..2e8b52b795
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v10 14/14] net/idpf: add support for timestamp offload
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-24 13:01 ` [PATCH v10 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-24 13:01 ` Junfeng Guo
13 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:01 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 47c686762d..0a0ffc2c29 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -12,5 +12,6 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 21315866bf..bd33fd1797 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -21,6 +21,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -100,7 +102,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 2485b3a784..d6080aff81 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -185,6 +185,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index abef84b3b0..90b2111781 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -941,6 +943,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1034,6 +1054,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1396,6 +1423,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1405,9 +1433,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1419,6 +1449,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
(volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1474,6 +1507,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1775,18 +1820,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1794,6 +1843,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1853,6 +1905,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index e808710b41..6e7fbaf7ef 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -41,6 +76,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -201,4 +238,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 00/18] add support for idpf PMD in DPDK
2022-10-24 13:01 ` [PATCH v10 01/14] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 01/18] common/idpf: introduce common library Junfeng Guo
` (17 more replies)
0 siblings, 18 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for packet type get
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 17 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1223 +++++++
drivers/net/idpf/idpf_ethdev.h | 273 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2353 ++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 294 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1434 +++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15272 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 01/18] common/idpf: introduce common library
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 02/18] net/idpf: add support for device initialization Junfeng Guo
` (16 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8502 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 02/18] net/idpf: add support for device initialization
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 8:57 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 03/18] net/idpf: add Tx queue setup Junfeng Guo
` (15 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 10 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 825 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 207 +++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 470 ++++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1669 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..7a44b8b5e4
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,10 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Linux = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..4af790de37 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -160,6 +160,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..75695085f8
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,825 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->dev_capa = 0;
+ dev_info->rx_offload_capa = 0;
+
+ dev_info->tx_offload_capa = 0;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static uint16_t
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+#define IDPF_RSS_KEY_LEN 52
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX) {
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX) {
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ } else if (vport_info->chunks.chunks[i].type ==
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ }
+ }
+
+ vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0)
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = (struct idpf_adapter *)args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = (int *)args;
+ char *end;
+ int num;
+
+ num = strtoul(value, &end, 10);
+
+ if (num != 0 && num != 1) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0)
+ return adapter;
+ }
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..c0ae801fd5
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_osdep.h>
+#include <base/idpf_type.h>
+#include <base/idpf_devids.h>
+#include <base/idpf_prototype.h>
+#include <base/idpf_lan_txrx.h>
+#include <base/idpf_lan_pf_regs.h>
+#include <base/virtchnl.h>
+#include <base/virtchnl2.h>
+#include <base/virtchnl2_lan_desc.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+
+ bool stopped;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+extern struct idpf_adapter_list adapter_list;
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..ec72a8bba7
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,470 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
+ sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = (struct idpf_dma_mem *)rte_zmalloc(NULL,
+ sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl_version_info version;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.hsplit_caps =
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
+ VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
+
+ caps_msg.rsc_caps =
+ VIRTCHNL2_CAP_RSC_IPV4_TCP |
+ VIRTCHNL2_CAP_RSC_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSC_IPV6_TCP |
+ VIRTCHNL2_CAP_RSC_IPV6_SCTP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_RDMA |
+ VIRTCHNL2_CAP_SRIOV |
+ VIRTCHNL2_CAP_MACFILTER |
+ VIRTCHNL2_CAP_FLOW_DIRECTOR |
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -1;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 03/18] net/idpf: add Tx queue setup
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 9:40 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 04/18] net/idpf: add Rx " Junfeng Guo
` (14 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 18 +-
drivers/net/idpf/idpf_rxtx.c | 371 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 76 +++++++
drivers/net/idpf/meson.build | 1 +
4 files changed, 464 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 75695085f8..c0307128be 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -10,6 +10,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -36,6 +37,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -54,11 +56,23 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
- dev_info->dev_capa = 0;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
dev_info->rx_offload_capa = 0;
-
dev_info->tx_offload_capa = 0;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..df0ed772c1
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,371 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static inline int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static inline void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static inline void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c5c43c3033
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define IDPF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 04/18] net/idpf: add Rx queue setup
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 03/18] net/idpf: add Tx queue setup Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 05/18] net/idpf: add support for device start and stop Junfeng Guo
` (13 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 16 +-
drivers/net/idpf/idpf_rxtx.c | 427 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 488 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index c0307128be..1d2075f466 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -37,6 +37,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -56,16 +57,29 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
- dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
dev_info->rx_offload_capa = 0;
dev_info->tx_offload_capa = 0;
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
.offloads = 0,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index df0ed772c1..76669504c3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static inline int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static inline int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,95 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static inline void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static inline void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
+static inline void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static inline void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +249,329 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_deferred_start = rx_conf->rx_deferred_start;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid", nb_desc);
+ return -EINVAL;
+ }
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, rx_conf, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
+ nb_desc > IDPF_MAX_RING_DESC ||
+ nb_desc < IDPF_MIN_RING_DESC) {
+ PMD_INIT_LOG(ERR, "Number (%u) of receive descriptors is invalid",
+ nb_desc);
+ return -EINVAL;
+ }
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c5c43c3033..a79e9a0b26 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,11 +15,53 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -69,6 +111,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 05/18] net/idpf: add support for device start and stop
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 04/18] net/idpf: add Rx " Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 9:49 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 06/18] net/idpf: add support for queue start Junfeng Guo
` (12 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 89 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 5 ++
2 files changed, 94 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1d2075f466..4c7a2d0748 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,17 +29,42 @@ static const char * const idpf_valid_args[] = {
};
static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
+ RTE_ETH_LINK_DOWN;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
@@ -233,6 +258,70 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+
+ PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
+ return -ENOTSUP;
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+
+ PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
+ return -ENOTSUP;
+ }
+
+ return err;
+}
+
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -1;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ if (idpf_start_queues(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return -1;
+ }
+
+ if (idpf_vc_ena_dis_vport(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index c0ae801fd5..070531cc48 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -105,6 +105,9 @@ struct idpf_vport {
/* Chunk info */
struct idpf_chunks_info chunks_info;
+ /* Event from ipf */
+ bool link_up;
+
uint16_t devarg_id;
};
@@ -195,6 +198,8 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
}
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+int idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 06/18] net/idpf: add support for queue start
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 07/18] net/idpf: add support for queue stop Junfeng Guo
` (11 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 18 +-
drivers/net/idpf/idpf_ethdev.h | 7 +
drivers/net/idpf/idpf_rxtx.c | 214 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 4 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 684 insertions(+), 6 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4c7a2d0748..954bd0bf4b 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -270,18 +272,22 @@ idpf_start_queues(struct rte_eth_dev *dev)
txq = dev->data->tx_queues[i];
if (txq == NULL || txq->tx_deferred_start)
continue;
-
- PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
- return -ENOTSUP;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
}
for (i = 0; i < dev->data->nb_rx_queues; i++) {
rxq = dev->data->rx_queues[i];
if (rxq == NULL || rxq->rx_deferred_start)
continue;
-
- PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
- return -ENOTSUP;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
}
return err;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 070531cc48..094f123be1 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -205,6 +205,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 76669504c3..b71c1ac6db 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -796,3 +796,217 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ (struct idpf_rx_queue *)dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ (struct idpf_tx_queue *)dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index a79e9a0b26..d076903e3c 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -115,8 +115,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ec72a8bba7..5e5508e999 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_CTLQ_LEN 64
@@ -227,6 +228,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -444,6 +449,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 07/18] net/idpf: add support for queue stop
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 06/18] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 08/18] net/idpf: add queue release Junfeng Guo
` (10 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++-
drivers/net/idpf/idpf_rxtx.c | 140 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 11 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
5 files changed, 232 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 7a44b8b5e4..30e1c0831e 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -7,4 +7,5 @@
; is selected.
;
[Features]
+Queue start/stop = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 954bd0bf4b..6040050dd9 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -62,7 +62,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -300,22 +302,26 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -1;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return -1;
+ goto err_mtu;
}
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- return -1;
+ goto err_vport;
}
return 0;
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return -1;
}
static int
@@ -325,6 +331,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
+ idpf_stop_queues(dev);
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index b71c1ac6db..c14fd4324e 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static inline void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static inline void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static inline void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -311,6 +360,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -552,6 +602,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -654,6 +705,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -778,6 +830,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -1010,3 +1063,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index d076903e3c..e1de436a4a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -111,16 +111,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 5e5508e999..9138799989 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -891,6 +891,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 08/18] net/idpf: add queue release
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 07/18] net/idpf: add support for queue stop Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 09/18] net/idpf: add support for packet type get Junfeng Guo
` (9 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6040050dd9..4c058660b4 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -66,7 +66,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index c14fd4324e..298eaf0a1a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -120,6 +120,51 @@ static const struct idpf_txq_ops def_txq_ops = {
.release_mbufs = release_txq_mbufs,
};
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = (struct idpf_rx_queue *)rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = (struct idpf_tx_queue *)txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -401,6 +446,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -541,6 +592,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -654,6 +711,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -776,6 +839,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1124,6 +1193,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index e1de436a4a..9f767079b2 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -126,12 +126,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 09/18] net/idpf: add support for packet type get
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 08/18] net/idpf: add queue release Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 9:57 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 10/18] net/idpf: add support for MTU configuration Junfeng Guo
` (8 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add dev ops dev_supported_ptypes_get.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 7 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 19 +++
drivers/net/idpf/idpf_rxtx.h | 7 +
drivers/net/idpf/idpf_vchnl.c | 240 ++++++++++++++++++++++++++++++
6 files changed, 280 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 30e1c0831e..a03068df85 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,4 +8,5 @@
;
[Features]
Queue start/stop = Y
+Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 4c058660b4..80826569c8 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -57,6 +57,7 @@ idpf_dev_link_update(struct rte_eth_dev *dev,
}
static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_supported_ptypes_get = idpf_dev_supported_ptypes_get,
.dev_configure = idpf_dev_configure,
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
@@ -641,6 +642,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 094f123be1..58f070c976 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -47,6 +47,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -141,6 +143,8 @@ struct idpf_adapter {
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
bool stopped;
};
@@ -202,6 +206,7 @@ int idpf_dev_link_update(struct rte_eth_dev *dev,
__rte_unused int wait_to_complete);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -213,6 +218,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 298eaf0a1a..c3c4acb69f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,25 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+const uint32_t *
+idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
+{
+ static const uint32_t ptypes[] = {
+ RTE_PTYPE_L2_ETHER,
+ RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
+ RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
+ RTE_PTYPE_L4_FRAG,
+ RTE_PTYPE_L4_NONFRAG,
+ RTE_PTYPE_L4_UDP,
+ RTE_PTYPE_L4_TCP,
+ RTE_PTYPE_L4_SCTP,
+ RTE_PTYPE_L4_ICMP,
+ RTE_PTYPE_UNKNOWN
+ };
+
+ return ptypes;
+}
+
static inline int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 9f767079b2..11f1fde5d4 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,10 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,4 +141,7 @@ int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9138799989..86a55d2ff1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -238,6 +238,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -286,6 +291,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return err;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -984,3 +1198,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 10/18] net/idpf: add support for MTU configuration
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 09/18] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 11/18] net/idpf: add support for basic Rx datapath Junfeng Guo
` (7 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 15 +++++++++++++++
2 files changed, 16 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index a03068df85..d4eb9b374c 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,5 +8,6 @@
;
[Features]
Queue start/stop = Y
+MTU update = Y
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 80826569c8..1654af0053 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -34,6 +34,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
int
@@ -72,6 +73,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -122,6 +124,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -209,6 +223,7 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->num_tx_complq = vport_info->num_tx_complq;
vport->num_rx_q = vport_info->num_rx_q;
vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
vport->sw_idx = idx;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 11/18] net/idpf: add support for basic Rx datapath
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 10/18] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 12/18] net/idpf: add support for basic Tx datapath Junfeng Guo
` (6 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 286 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 5 +
3 files changed, 293 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1654af0053..f03df0a3ef 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -330,6 +330,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index c3c4acb69f..4037cf5a74 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1250,3 +1250,289 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring =
+ (volatile struct virtchnl2_splitq_rx_buf_desc *)rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = (struct idpf_rx_queue *)rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring =
+ (volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 11f1fde5d4..056596ce94 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -140,8 +140,13 @@ int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+void idpf_set_rx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 12/18] net/idpf: add support for basic Tx datapath
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 11/18] net/idpf: add support for basic Rx datapath Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 10:12 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (5 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 353 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 8 +
4 files changed, 364 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f03df0a3ef..d3ac269419 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -94,7 +94,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
dev_info->rx_offload_capa = 0;
- dev_info->tx_offload_capa = 0;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
@@ -331,6 +331,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 58f070c976..238c641c3f 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -44,6 +44,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4037cf5a74..fd203c985d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1415,6 +1415,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1526,6 +1668,204 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ ret = rte_net_intel_cksum_prepare(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1536,3 +1876,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 056596ce94..eb0b230d3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -144,9 +144,17 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
+
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 13/18] net/idpf: add support for write back based on ITR expire
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 12/18] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 14/18] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 111 +++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d3ac269419..57e4bf0d67 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -280,6 +280,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps != 0) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -317,6 +401,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
@@ -325,6 +413,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -354,6 +459,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -365,6 +474,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -717,6 +831,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 238c641c3f..d89a06d239 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -42,6 +42,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -105,6 +108,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -140,6 +148,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -219,6 +229,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 86a55d2ff1..34369cfc96 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -1024,6 +1028,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 14/18] net/idpf: add support for RSS
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 15/18] net/idpf: add support for Rx offloading Junfeng Guo
` (3 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 ++++++++
drivers/net/idpf/idpf_vchnl.c | 97 +++++++++++++++++++++++++++
3 files changed, 237 insertions(+), 2 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 57e4bf0d67..739cf31d65 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -90,6 +90,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
@@ -226,6 +227,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -271,13 +276,114 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = (uint8_t *)rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = (uint32_t *)rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
- if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0)
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret = 0;
+
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0) {
PMD_INIT_LOG(ERR, "RSS is not supported.");
+ }
- return 0;
+ return ret;
}
static int
@@ -474,6 +580,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index d89a06d239..7fe2647956 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -57,6 +57,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
@@ -103,11 +117,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -221,6 +244,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 34369cfc96..63605f2f64 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -228,6 +228,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -667,6 +670,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 15/18] net/idpf: add support for Rx offloading
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (13 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 14/18] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 10:03 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 16/18] net/idpf: add support for Tx offloading Junfeng Guo
` (2 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 2 +
drivers/net/idpf/idpf_ethdev.c | 9 ++-
drivers/net/idpf/idpf_rxtx.c | 122 ++++++++++++++++++++++++++++++
3 files changed, 132 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d4eb9b374c..c86d9378ea 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,5 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 739cf31d65..d8cc423a23 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -94,7 +94,14 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
- dev_info->rx_offload_capa = 0;
+
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index fd203c985d..143c8b69f3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1250,6 +1250,72 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
@@ -1325,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1393,11 +1461,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1557,6 +1632,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1590,6 +1707,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1655,10 +1773,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 16/18] net/idpf: add support for Tx offloading
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (14 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 15/18] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-25 10:14 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Tx offloading support:
- support TSO
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 134 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 158 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c86d9378ea..47c686762d 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,6 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Packet type parsing = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d8cc423a23..21315866bf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -102,7 +102,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_RSS_HASH;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 143c8b69f3..8f82cf1b59 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1549,6 +1549,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1557,11 +1600,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1591,7 +1637,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1842,14 +1910,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1876,11 +1947,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1908,6 +1987,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1968,16 +2070,44 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
int i, ret;
+ uint64_t ol_flags;
struct rte_mbuf *m;
for (i = 0; i < nb_pkts; i++) {
m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
+ }
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
ret = rte_net_intel_cksum_prepare(m);
if (ret != 0) {
rte_errno = -ret;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index eb0b230d3a..efb2734d85 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,16 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (15 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 16/18] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 22 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
7 files changed, 1190 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7fe2647956..2485b3a784 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -180,6 +180,11 @@ struct idpf_adapter {
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
bool stopped;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8f82cf1b59..abef84b3b0 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
@@ -271,6 +273,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static inline void
@@ -2118,25 +2122,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index efb2734d85..e808710b41 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -52,6 +58,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -82,6 +93,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -166,12 +181,19 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..2e8b52b795
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v11 18/18] net/idpf: add support for timestamp offload
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (16 preceding siblings ...)
2022-10-24 13:12 ` [PATCH v11 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-24 13:12 ` Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-24 13:12 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 47c686762d..0a0ffc2c29 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -12,5 +12,6 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Packet type parsing = Y
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 21315866bf..bd33fd1797 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -21,6 +21,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -100,7 +102,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
- RTE_ETH_RX_OFFLOAD_RSS_HASH;
+ RTE_ETH_RX_OFFLOAD_RSS_HASH |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 2485b3a784..d6080aff81 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -185,6 +185,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index abef84b3b0..90b2111781 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
const uint32_t *
idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
{
@@ -941,6 +943,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -1034,6 +1054,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1396,6 +1423,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1405,9 +1433,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = (struct idpf_rx_queue *)rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1419,6 +1449,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
(volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *)rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1474,6 +1507,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1775,18 +1820,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1794,6 +1843,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1853,6 +1905,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index e808710b41..6e7fbaf7ef 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -41,6 +76,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -201,4 +238,57 @@ void idpf_set_tx_function(struct rte_eth_dev *dev);
const uint32_t *idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading
2022-10-21 8:29 ` Andrew Rybchenko
@ 2022-10-24 13:26 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-24 13:26 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun
> > +
> > +/* Translate the rx descriptor status and error fields to pkt flags
> > +*/ static inline uint64_t idpf_rxd_to_pkt_flags(uint16_t
> > +status_error) {
> > + uint64_t flags = 0;
> > +
> > + if (unlikely(!(status_error &
> BIT(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S))))
> > + return flags;
> > +
> > + if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) ==
> 0)) {
> > + flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
>
> Strictly speaking description of the
> RTE_MBUF_F_RX_IP_CKSUM_GOOD says that IP checksum in the patcket is
> valid, but there is no checksum in IPv6 header.
> So, we can't say that it is valid for IPv6...
Hi Andrew,
Almost all the comments are addressed in V11, please help to review. Thanks.
Here, since device just writes back 0 or 1 in descriptor, can’t distinguish Ipv4 or IPv6, we set RTE_MBUF_F_RX_IP_CKSUM_GOOD here. It follows ice or iavf implementation.
BTW, there's no common code in common/idpf currently, only base folder, there'll be some comment codes shared by idpf and a new PMD in next release.
>
> > + RTE_MBUF_F_RX_L4_CKSUM_GOOD);
> > + return flags;
> > + }
> > +
> > + if (unlikely(status_error &
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v9 02/14] net/idpf: add support for device initialization
2022-10-21 12:41 ` Zhang, Qi Z
@ 2022-10-25 7:52 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 7:52 UTC (permalink / raw)
To: Zhang, Qi Z, Guo, Junfeng, Wu, Jingjing, Xing, Beilei
Cc: dev, Li, Xiaoyun, Wang, Xiao W
On 10/21/22 15:41, Zhang, Qi Z wrote:
> Hi Andrew:
>
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Friday, October 21, 2022 3:48 PM
>> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
>> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
>> <beilei.xing@intel.com>
>> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
>> <xiao.w.wang@intel.com>
>> Subject: Re: [PATCH v9 02/14] net/idpf: add support for device initialization
>>
>> On 10/21/22 10:39, Andrew Rybchenko wrote:
>>> On 10/21/22 08:18, Junfeng Guo wrote:
>>>> Support device init and add the following dev ops skeleton:
>>>> - dev_configure
>>>> - dev_start
>>
>> One more question: are you sure that you can start without
>> queues setup?
>
> I agree the patch set is not well-ordered. Keeping each increment be measurable is a good practice. We should keep this in mind.
> But not sure if this is a blocking issue for patch merging as we saw the risk to have this in RC2 if we must to rework for this.
> Btw, the first version of idpf PMD is quite simple. It's very similar to iavf PMD but has a few features,
I see. IMHO there are too many very similar Intel drivers which
just blow up code base. Just my opinion. Nothing else.
> so I hope the out-of-order issue may not downgrade the readability :)
> Please kindly give your suggestion.
>
> Thanks
> Qi
>
>
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 02/18] net/idpf: add support for device initialization
2022-10-24 13:12 ` [PATCH v11 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-25 8:57 ` Andrew Rybchenko
2022-10-26 8:28 ` Xing, Beilei
2022-10-28 15:14 ` Andrew Rybchenko
0 siblings, 2 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 8:57 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang
On 10/24/22 16:12, Junfeng Guo wrote:
> Support device init and add the following dev ops:
> - dev_configure
> - dev_close
> - dev_infos_get
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> new file mode 100644
> index 0000000000..7a44b8b5e4
> --- /dev/null
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -0,0 +1,10 @@
> +;
> +; Supported features of the 'idpf' network poll mode driver.
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +; A feature with "P" indicates only be supported when non-vector path
> +; is selected.
The statement should be added when the first P appears.
> +;
> +[Features]
> +Linux = Y
I think you should add at least one architecture.
I guess it should be x86_64 since otherwise it is unclear
where to build...
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> new file mode 100644
> index 0000000000..75695085f8
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -0,0 +1,825 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#define IDPF_TX_SINGLE_Q "tx_single"
> +#define IDPF_RX_SINGLE_Q "rx_single"
> +#define IDPF_VPORT "vport"
> +
> +rte_spinlock_t idpf_adapter_lock;
> +/* A list for all adapters, one adapter matches one PCI device */
> +struct idpf_adapter_list idpf_adapter_list;
> +bool idpf_adapter_list_init;
> +
> +static const char * const idpf_valid_args[] = {
> + IDPF_TX_SINGLE_Q,
> + IDPF_RX_SINGLE_Q,
> + IDPF_VPORT,
> + NULL
> +};
> +
> +static int idpf_dev_configure(struct rte_eth_dev *dev);
> +static int idpf_dev_close(struct rte_eth_dev *dev);
> +static int idpf_dev_info_get(struct rte_eth_dev *dev,
> + struct rte_eth_dev_info *dev_info);
> +static void idpf_adapter_rel(struct idpf_adapter *adapter);
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_close = idpf_dev_close,
> + .dev_infos_get = idpf_dev_info_get,
> +};
> +
> +static int
> +idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + dev_info->max_rx_queues = adapter->caps->max_rx_q;
> + dev_info->max_tx_queues = adapter->caps->max_tx_q;
> + dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
> + dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
> +
> + dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
> + dev_info->min_mtu = RTE_ETHER_MIN_MTU;
> +
> + dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> + dev_info->dev_capa = 0;
> + dev_info->rx_offload_capa = 0;
> +
> + dev_info->tx_offload_capa = 0;
You don't need above zeros since the structure is memset to 0
in ethdev layer before driver callback invocation.
> +
> + return 0;
> +}
> +
> +static int
> +idpf_init_vport_req_info(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_create_vport *vport_info;
> + uint16_t idx = adapter->cur_vport_idx;
> +
> + if (idx == IDPF_INVALID_VPORT_IDX) {
> + PMD_INIT_LOG(ERR, "Invalid vport index.");
> + return -1;
> + }
> +
> + if (adapter->vport_req_info[idx] == NULL) {
> + adapter->vport_req_info[idx] = rte_zmalloc(NULL,
> + sizeof(struct virtchnl2_create_vport), 0);
> + if (adapter->vport_req_info[idx] == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
> + return -1;
> + }
> + }
> +
> + vport_info =
> + (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
> +
> + vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
> + if (adapter->txq_model == 0) {
> + vport_info->txq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
> + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
> + vport_info->num_tx_complq =
> + IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
> + } else {
> + vport_info->txq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
> + vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
> + vport_info->num_tx_complq = 0;
> + }
> + if (adapter->rxq_model == 0) {
> + vport_info->rxq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
> + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
> + vport_info->num_rx_bufq =
> + IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
> + } else {
> + vport_info->rxq_model =
> + rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
> + vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
> + vport_info->num_rx_bufq = 0;
> + }
> +
> + return 0;
> +}
> +
> +static uint16_t
> +idpf_parse_devarg_id(char *name)
> +{
> + uint16_t val;
> + char *p;
> +
> + p = strstr(name, "vport_");
> + p += sizeof("vport_") - 1;
> +
> + val = strtoul(p, NULL, 10);
There is a potential segfault here if there is no vport_ in the
name.
> +
> + return val;
> +}
> +
> +#define IDPF_RSS_KEY_LEN 52
The define is not used. It must be a part of the RSS support
patch and it should be use to set key size in device
information.
> +
> +static int
> +idpf_init_vport(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t idx = adapter->cur_vport_idx;
> + struct virtchnl2_create_vport *vport_info =
> + (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
> + int i;
> +
> + vport->vport_id = vport_info->vport_id;
> + vport->txq_model = vport_info->txq_model;
> + vport->rxq_model = vport_info->rxq_model;
> + vport->num_tx_q = vport_info->num_tx_q;
> + vport->num_tx_complq = vport_info->num_tx_complq;
> + vport->num_rx_q = vport_info->num_rx_q;
> + vport->num_rx_bufq = vport_info->num_rx_bufq;
> + rte_memcpy(vport->default_mac_addr,
> + vport_info->default_mac_addr, ETH_ALEN);
> + vport->sw_idx = idx;
> +
> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> + if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX) {
switch/case will be more readable here
> + vport->chunks_info.tx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX) {
> + vport->chunks_info.rx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
> + vport->chunks_info.tx_compl_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_compl_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_compl_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
> + vport->chunks_info.rx_buf_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_buf_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_buf_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + }
> + }
> +
> + vport->devarg_id = idpf_parse_devarg_id(dev->data->name);
> + vport->dev_data = dev->data;
> +
> + adapter->vports[idx] = vport;
> +
> + return 0;
> +}
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev)
> +{
> + if ((dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG) != 0)
> + PMD_INIT_LOG(ERR, "RSS is not supported.");
Do you think it is really enought?
You ignore:
- link_speeds
- rxmode in the case of DCB-only and/or with VMDq
- txmode in DCB and VMQq cases
- txmode: pvid, hw_vlan*
- lpbk_mode
- dcb_capability_en
- intr_conf
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_close(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + idpf_vc_destroy_vport(vport);
> +
> +
> + adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
> +
> + rte_free(vport);
> + dev->data->dev_private = NULL;
> +
> + return 0;
> +}
> +
> +static int
> +insert_value(struct idpf_adapter *adapter, uint16_t id)
> +{
> + uint16_t i;
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if (adapter->req_vports[i] == id)
> + return 0;
> + }
> +
> + if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + adapter->req_vports[adapter->req_vport_nb] = id;
> + adapter->req_vport_nb++;
> +
> + return 0;
> +}
> +
> +static const char *
> +parse_range(const char *value, struct idpf_adapter *adapter)
> +{
> + uint16_t lo, hi, i;
> + int n = 0;
> + int result;
> + const char *pos = value;
> +
> + result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
> + if (result == 1) {
> + if (lo >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + if (insert_value(adapter, lo) != 0)
> + return NULL;
> + } else if (result == 2) {
> + if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
> + return NULL;
> + for (i = lo; i <= hi; i++) {
> + if (insert_value(adapter, i) != 0)
> + return NULL;
> + }
> + } else {
> + return NULL;
> + }
> +
> + return pos + n;
> +}
> +
> +static int
> +parse_vport(const char *key, const char *value, void *args)
> +{
> + struct idpf_adapter *adapter = (struct idpf_adapter *)args;
type cast from void * is not required in C
> + const char *pos = value;
> + int i;
> +
> + adapter->req_vport_nb = 0;
> +
> + if (*pos == '[')
> + pos++;
> +
> + while (1) {
> + pos = parse_range(pos, adapter);
> + if (pos == NULL) {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> + if (*pos != ',')
> + break;
> + pos++;
> + }
> +
> + if (*value == '[' && *pos != ']') {
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
> + value, key);
> + return -1;
> + }
> +
> + if (adapter->cur_vport_nb + adapter->req_vport_nb >
> + IDPF_MAX_VPORT_NUM) {
> + PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
> + IDPF_MAX_VPORT_NUM);
> + return -1;
> + }
> +
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
> + adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
> + adapter->cur_vport_nb++;
> + } else {
> + PMD_INIT_LOG(ERR, "Vport %d has been created",
> + adapter->req_vports[i]);
> + return -1;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int
> +parse_bool(const char *key, const char *value, void *args)
> +{
> + int *i = (int *)args;
type cast from void * is not required in C
> + char *end;
> + int num;
> +
> + num = strtoul(value, &end, 10);
man strtoul:
The implementation may also set errno to EINVAL in case no conversion
was performed (no digits seen, and 0 returned).
> +
> + if (num != 0 && num != 1) {
So, 0 may be returned in the case of failure, but you treat it
as OK here.
> + PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
> + value, key);
> + return -1;
> + }
> +
> + *i = num;
> + return 0;
> +}
> +
> +static int
> +idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
> +{
> + struct rte_devargs *devargs = pci_dev->device.devargs;
> + struct rte_kvargs *kvlist;
> + int ret;
> +
> + if (devargs == NULL)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
> + if (kvlist == NULL) {
> + PMD_INIT_LOG(ERR, "invalid kvargs key");
> + return -EINVAL;
> + }
> +
> + ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
> + adapter);
> + if (ret != 0)
> + goto bail;
> +
> + ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
> + &adapter->txq_model);
> + if (ret != 0)
> + goto bail;
> +
> + ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
> + &adapter->rxq_model);
> + if (ret != 0)
> + goto bail;
> +
> +bail:
> + rte_kvargs_free(kvlist);
> + return ret;
> +}
> +
> +static void
> +idpf_reset_pf(struct idpf_hw *hw)
> +{
> + uint32_t reg;
> +
> + reg = IDPF_READ_REG(hw, PFGEN_CTRL);
> + IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
> +}
> +
> +#define IDPF_RESET_WAIT_CNT 100
> +static int
> +idpf_check_pf_reset_done(struct idpf_hw *hw)
> +{
> + uint32_t reg;
> + int i;
> +
> + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
> + reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
> + if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
> + return 0;
> + rte_delay_ms(1000);
> + }
> +
> + PMD_INIT_LOG(ERR, "IDPF reset timeout");
> + return -EBUSY;
> +}
> +
> +#define CTLQ_NUM 2
> +static int
> +idpf_init_mbx(struct idpf_hw *hw)
> +{
> + struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
> + {
> + .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ATQH,
> + .tail = PF_FW_ATQT,
> + .len = PF_FW_ATQLEN,
> + .bah = PF_FW_ATQBAH,
> + .bal = PF_FW_ATQBAL,
> + .len_mask = PF_FW_ATQLEN_ATQLEN_M,
> + .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
> + .head_mask = PF_FW_ATQH_ATQH_M,
> + }
> + },
> + {
> + .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ARQH,
> + .tail = PF_FW_ARQT,
> + .len = PF_FW_ARQLEN,
> + .bah = PF_FW_ARQBAH,
> + .bal = PF_FW_ARQBAL,
> + .len_mask = PF_FW_ARQLEN_ARQLEN_M,
> + .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
> + .head_mask = PF_FW_ARQH_ARQH_M,
> + }
> + }
> + };
> + struct idpf_ctlq_info *ctlq;
> + int ret;
> +
> + ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
> + if (ret != 0)
> + return ret;
> +
> + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
> + struct idpf_ctlq_info, cq_list) {
> + if (ctlq->q_id == IDPF_CTLQ_ID &&
> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
> + hw->asq = ctlq;
> + if (ctlq->q_id == IDPF_CTLQ_ID &&
> + ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
> + hw->arq = ctlq;
> + }
> +
> + if (hw->asq == NULL || hw->arq == NULL) {
> + idpf_ctlq_deinit(hw);
> + ret = -ENOENT;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
> +{
> + struct idpf_hw *hw = &adapter->hw;
> + int ret = 0;
> +
> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
> + hw->back = adapter;
> + hw->vendor_id = pci_dev->id.vendor_id;
> + hw->device_id = pci_dev->id.device_id;
> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
> +
> + strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
> +
> + idpf_reset_pf(hw);
> + ret = idpf_check_pf_reset_done(hw);
> + if (ret != 0) {
> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
> + goto err;
> + }
> +
> + ret = idpf_init_mbx(hw);
> + if (ret != 0) {
> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
> + goto err;
> + }
> +
> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
> + IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (adapter->mbx_resp == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
> + goto err_mbx;
> + }
> +
> + if (idpf_vc_check_api_version(adapter) != 0) {
> + PMD_INIT_LOG(ERR, "Failed to check api version");
> + goto err_api;
> + }
> +
> + adapter->caps = rte_zmalloc("idpf_caps",
> + sizeof(struct virtchnl2_get_capabilities), 0);
> + if (adapter->caps == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
> + goto err_api;
> + }
> +
> + if (idpf_vc_get_caps(adapter) != 0) {
> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
> + goto err_caps;
> + }
> +
> + adapter->max_vport_nb = adapter->caps->max_vports;
> +
> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_req_info),
> + 0);
> + if (adapter->vport_req_info == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
> + goto err_caps;
> + }
> +
> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vport_recv_info),
> + 0);
> + if (adapter->vport_recv_info == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
> + goto err_vport_recv_info;
> + }
> +
> + adapter->vports = rte_zmalloc("vports",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vports),
> + 0);
> + if (adapter->vports == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
> + goto err_vports;
> + }
> +
> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_rx_queues)) /
> + sizeof(struct virtchnl2_rxq_info);
> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_tx_queues)) /
> + sizeof(struct virtchnl2_txq_info);
> +
> + adapter->cur_vports = 0;
> + adapter->cur_vport_nb = 0;
> +
> + return ret;
> +
> +err_vports:
> + rte_free(adapter->vport_recv_info);
> + adapter->vport_recv_info = NULL;
> +err_vport_recv_info:
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> +err_caps:
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +err_api:
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +err_mbx:
> + idpf_ctlq_deinit(hw);
> +err:
> + return -1;
> +}
> +
> +static uint16_t
> +idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
> +{
> + uint16_t vport_idx;
> + uint16_t i;
> +
> + for (i = 0; i < max_vport_nb; i++) {
> + if (vports[i] == NULL)
> + break;
> + }
> +
> + if (i == max_vport_nb)
> + vport_idx = IDPF_INVALID_VPORT_IDX;
> + else
> + vport_idx = i;
> +
> + return vport_idx;
> +}
> +
> +static int
> +idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = init_params;
> + int ret = 0;
> +
> + dev->dev_ops = &idpf_eth_dev_ops;
> + vport->adapter = adapter;
> +
> + ret = idpf_init_vport_req_info(dev);
> + if (ret != 0) {
> + PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
> + goto err;
> + }
> +
> + ret = idpf_vc_create_vport(adapter);
> + if (ret != 0) {
> + PMD_INIT_LOG(ERR, "Failed to create vport.");
> + goto err_create_vport;
> + }
> +
> + ret = idpf_init_vport(dev);
> + if (ret != 0) {
> + PMD_INIT_LOG(ERR, "Failed to init vports.");
> + goto err_init_vport;
> + }
> +
> + adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
> + adapter->max_vport_nb);
> +
> + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
> + if (dev->data->mac_addrs == NULL) {
> + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
> + ret = -ENOMEM;
> + goto err_init_vport;
> + }
> +
> + rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
> + &dev->data->mac_addrs[0]);
> +
> + return 0;
> +
> +err_init_vport:
> + idpf_vc_destroy_vport(vport);
> +err_create_vport:
> + rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
> +err:
> + return ret;
> +}
> +
> +static const struct rte_pci_id pci_id_idpf_map[] = {
> + { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +struct idpf_adapter *
> +idpf_find_adapter(struct rte_pci_device *pci_dev)
It looks like the function requires corresponding lock to be
held. If yes, it should be documented and code fixed. If no,
it should be explaiend why.
> +{
> + struct idpf_adapter *adapter;
> +
> + TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
> + if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0)
> + return adapter;
> + }
> +
> + return NULL;
> +}
> +
> +static int
> +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter;
> + char name[RTE_ETH_NAME_MAX_LEN];
> + int i, retval;
> + bool first_probe = false;
> +
> + if (!idpf_adapter_list_init) {
> + rte_spinlock_init(&idpf_adapter_lock);
> + TAILQ_INIT(&idpf_adapter_list);
> + idpf_adapter_list_init = true;
> + }
> +
> + adapter = idpf_find_adapter(pci_dev);
> + if (adapter == NULL) {
> + first_probe = true;
> + adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
> + sizeof(struct idpf_adapter), 0);
> + if (adapter == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
> + return -1;
> + }
> +
> + retval = idpf_adapter_init(pci_dev, adapter);
> + if (retval != 0) {
> + PMD_INIT_LOG(ERR, "Failed to init adapter.");
> + return retval;
> + }
> +
> + rte_spinlock_lock(&idpf_adapter_lock);
> + TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
> + rte_spinlock_unlock(&idpf_adapter_lock);
> + }
> +
> + retval = idpf_parse_devargs(pci_dev, adapter);
> + if (retval != 0) {
> + PMD_INIT_LOG(ERR, "Failed to parse private devargs");
> + goto err;
> + }
> +
> + if (adapter->req_vport_nb == 0) {
> + /* If no vport devarg, create vport 0 by default. */
> + snprintf(name, sizeof(name), "idpf_%s_vport_0",
> + pci_dev->device.name);
> + retval = rte_eth_dev_create(&pci_dev->device, name,
> + sizeof(struct idpf_vport),
> + NULL, NULL, idpf_dev_init,
> + adapter);
> + if (retval != 0)
> + PMD_DRV_LOG(ERR, "Failed to create default vport 0");
> + adapter->cur_vports |= RTE_BIT32(0);
> + adapter->cur_vport_nb++;
> + } else {
> + for (i = 0; i < adapter->req_vport_nb; i++) {
> + snprintf(name, sizeof(name), "idpf_%s_vport_%d",
> + pci_dev->device.name,
> + adapter->req_vports[i]);
> + retval = rte_eth_dev_create(&pci_dev->device, name,
> + sizeof(struct idpf_vport),
> + NULL, NULL, idpf_dev_init,
> + adapter);
> + if (retval != 0)
> + PMD_DRV_LOG(ERR, "Failed to create vport %d",
> + adapter->req_vports[i]);
> + }
> + }
> +
> + return 0;
> +
> +err:
> + if (first_probe) {
> + rte_spinlock_lock(&idpf_adapter_lock);
> + TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
> + rte_spinlock_unlock(&idpf_adapter_lock);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> + }
> + return retval;
> +}
> +
> +static void
> +idpf_adapter_rel(struct idpf_adapter *adapter)
> +{
> + struct idpf_hw *hw = &adapter->hw;
> + int i;
> +
> + idpf_ctlq_deinit(hw);
> +
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +
> + if (adapter->vport_req_info != NULL) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_req_info[i]);
> + adapter->vport_req_info[i] = NULL;
> + }
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> + }
> +
> + if (adapter->vport_recv_info != NULL) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + rte_free(adapter->vport_recv_info[i]);
> + adapter->vport_recv_info[i] = NULL;
> + }
> + rte_free(adapter->vport_recv_info);
> + adapter->vport_recv_info = NULL;
> + }
> +
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +}
> +
> +static int
> +idpf_pci_remove(struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
> + uint16_t port_id;
> +
> + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
> + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
> + rte_eth_dev_close(port_id);
> + }
> +
> + rte_spinlock_lock(&idpf_adapter_lock);
> + TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
> + rte_spinlock_unlock(&idpf_adapter_lock);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> +
> + return 0;
> +}
> +
> +static struct rte_pci_driver rte_idpf_pmd = {
> + .id_table = pci_id_idpf_map,
> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
> + .probe = idpf_pci_probe,
> + .remove = idpf_pci_remove,
> +};
> +
> +/**
> + * Driver initialization routine.
> + * Invoked once at EAL init time.
> + * Register itself as the [Poll Mode] Driver of PCI devices.
> + */
> +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
> +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
> +
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> new file mode 100644
> index 0000000000..c0ae801fd5
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -0,0 +1,207 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_ETHDEV_H_
> +#define _IDPF_ETHDEV_H_
> +
> +#include <stdint.h>
> +#include <rte_mbuf.h>
seems to be unused
> +#include <rte_mempool.h>
seems to be unused
> +#include <rte_malloc.h>
> +#include <rte_spinlock.h>
> +#include <rte_ethdev.h>
> +#include <rte_kvargs.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +
> +#include "idpf_logs.h"
> +
> +#include <base/idpf_osdep.h>
> +#include <base/idpf_type.h>
> +#include <base/idpf_devids.h>
> +#include <base/idpf_prototype.h>
> +#include <base/idpf_lan_txrx.h>
> +#include <base/idpf_lan_pf_regs.h>
> +#include <base/virtchnl.h>
> +#include <base/virtchnl2.h>
> +#include <base/virtchnl2_lan_desc.h>
Are you sure that everything above used in the
patch?
> +
> +#define IDPF_MAX_VPORT_NUM 8
> +
> +#define IDPF_DEFAULT_RXQ_NUM 16
> +#define IDPF_DEFAULT_TXQ_NUM 16
> +
> +#define IDPF_INVALID_VPORT_IDX 0xffff
> +#define IDPF_TXQ_PER_GRP 1
unused
> +#define IDPF_TX_COMPLQ_PER_GRP 1
> +#define IDPF_RXQ_PER_GRP 1
unused
> +#define IDPF_RX_BUFQ_PER_GRP 2
> +
> +#define IDPF_CTLQ_ID -1
> +#define IDPF_CTLQ_LEN 64
> +#define IDPF_DFLT_MBX_BUF_SIZE 4096
> +
> +#define IDPF_MIN_BUF_SIZE 1024
> +#define IDPF_MAX_FRAME_SIZE 9728
> +
> +#define IDPF_NUM_MACADDR_MAX 64
> +
> +#define IDPF_VLAN_TAG_SIZE 4
> +#define IDPF_ETH_OVERHEAD \
> + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
> +
> +#ifndef ETH_ADDR_LEN
> +#define ETH_ADDR_LEN 6
> +#endif
Why do you need it?
> +
> +#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
> +
> +/* Message type read in virtual channel from PF */
> +enum idpf_vc_result {
> + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
> + IDPF_MSG_NON, /* Read nothing from admin queue */
> + IDPF_MSG_SYS, /* Read system msg from admin queue */
> + IDPF_MSG_CMD, /* Read async command result */
> +};
> +
> +struct idpf_chunks_info {
> + uint32_t tx_start_qid;
> + uint32_t rx_start_qid;
> + /* Valid only if split queue model */
> + uint32_t tx_compl_start_qid;
> + uint32_t rx_buf_start_qid;
> +
> + uint64_t tx_qtail_start;
> + uint32_t tx_qtail_spacing;
> + uint64_t rx_qtail_start;
> + uint32_t rx_qtail_spacing;
> + uint64_t tx_compl_qtail_start;
> + uint32_t tx_compl_qtail_spacing;
> + uint64_t rx_buf_qtail_start;
> + uint32_t rx_buf_qtail_spacing;
> +};
> +
> +struct idpf_vport {
> + struct idpf_adapter *adapter; /* Backreference to associated adapter */
> + uint16_t vport_id;
> + uint32_t txq_model;
> + uint32_t rxq_model;
> + uint16_t num_tx_q;
Shouldn't it be set as the result of configure?
data->nb_tx_queues which is the result of the configure is not
used in the patch?
> + /* valid only if txq_model is split Q */
> + uint16_t num_tx_complq;
> + uint16_t num_rx_q;
same here.
> + /* valid only if rxq_model is split Q */
> + uint16_t num_rx_bufq;
> +
> + uint16_t max_mtu;
unused
> + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
> +
> + uint16_t sw_idx; /* SW idx */
> +
> + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
> + uint16_t max_pkt_len; /* Maximum packet length */
> +
> + /* Chunk info */
> + struct idpf_chunks_info chunks_info;
> +
> + uint16_t devarg_id;
> +};
> +
> +struct idpf_adapter {
> + TAILQ_ENTRY(idpf_adapter) next;
> + struct idpf_hw hw;
> + char name[IDPF_ADAPTER_NAME_LEN];
> +
> + struct virtchnl_version_info virtchnl_version;
unused. I'm stopping unused check here. Please, do it yourself
carefully after each patch to avoid more review iterations.
> + struct virtchnl2_get_capabilities *caps;
> +
> + volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
> + uint32_t cmd_retval; /* return value of the cmd response from ipf */
> + uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
> +
> + uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
> + uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
> +
> + /* Vport info */
> + uint8_t **vport_req_info;
> + uint8_t **vport_recv_info;
> + struct idpf_vport **vports;
> + uint16_t max_vport_nb;
> + uint16_t req_vports[IDPF_MAX_VPORT_NUM];
> + uint16_t req_vport_nb;
> + uint16_t cur_vports;
> + uint16_t cur_vport_nb;
> + uint16_t cur_vport_idx;
> +
> + /* Max config queue number per VC message */
> + uint32_t max_rxq_per_msg;
> + uint32_t max_txq_per_msg;
> +
> + bool stopped;
> +};
> +
> +TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
> +extern struct idpf_adapter_list adapter_list;
there is no adapter_list variable.
[snip]
> diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
> new file mode 100644
> index 0000000000..ec72a8bba7
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_vchnl.c
[snip]
> +static int
> +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
> + uint16_t msg_size, uint8_t *msg)
> +{
> + struct idpf_ctlq_msg *ctlq_msg;
> + struct idpf_dma_mem *dma_mem;
> + int err;
> +
> + err = idpf_vc_clean(adapter);
> + if (err != 0)
> + goto err;
> +
> + ctlq_msg = (struct idpf_ctlq_msg *)rte_zmalloc(NULL,
> + sizeof(struct idpf_ctlq_msg), 0);
Type cast from void * is not required in C. Fix it everywhere,
please.
[snip]
> +int
> +idpf_vc_get_caps(struct idpf_adapter *adapter)
> +{
> + struct virtchnl2_get_capabilities caps_msg;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> + caps_msg.csum_caps =
> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> +
> + caps_msg.seg_caps =
> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> + VIRTCHNL2_CAP_SEG_GENERIC;
> +
> + caps_msg.rss_caps =
> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> +
> + caps_msg.hsplit_caps =
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
> +
> + caps_msg.rsc_caps =
> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
> +
> + caps_msg.other_caps =
> + VIRTCHNL2_CAP_RDMA |
> + VIRTCHNL2_CAP_SRIOV |
> + VIRTCHNL2_CAP_MACFILTER |
> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> + VIRTCHNL2_CAP_CRC |
> + VIRTCHNL2_CAP_WB_ON_ITR |
> + VIRTCHNL2_CAP_PROMISC |
> + VIRTCHNL2_CAP_LINK_SPEED |
> + VIRTCHNL2_CAP_VLAN;
I'm wondering why all above capabilities are mentioned in the
patch? What does the API do? Do it is request it? Negotiage?
> +
> + args.ops = VIRTCHNL2_OP_GET_CAPS;
> + args.in_args = (uint8_t *)&caps_msg;
> + args.in_args_size = sizeof(caps_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err != 0) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
> + return err;
> + }
> +
> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> +
> + return 0;
> +}
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 03/18] net/idpf: add Tx queue setup
2022-10-24 13:12 ` [PATCH v11 03/18] net/idpf: add Tx queue setup Junfeng Guo
@ 2022-10-25 9:40 ` Andrew Rybchenko
2022-10-26 8:34 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 9:40 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/24/22 16:12, Junfeng Guo wrote:
> Add support for tx_queue_setup ops.
>
> In the single queue model, the same descriptor queue is used by SW to
> post buffer descriptors to HW and by HW to post completed descriptors
> to SW.
>
> In the split queue model, "RX buffer queues" are used to pass
> descriptor buffers from SW to HW while Rx queues are used only to
> pass the descriptor completions, that is, descriptors that point
> to completed buffers, from HW to SW. This is contrary to the single
> queue model in which Rx queues are used for both purposes.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index 75695085f8..c0307128be 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -10,6 +10,7 @@
> #include <rte_dev.h>
>
> #include "idpf_ethdev.h"
> +#include "idpf_rxtx.h"
>
> #define IDPF_TX_SINGLE_Q "tx_single"
> #define IDPF_RX_SINGLE_Q "rx_single"
> @@ -36,6 +37,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
> static const struct eth_dev_ops idpf_eth_dev_ops = {
> .dev_configure = idpf_dev_configure,
> .dev_close = idpf_dev_close,
> + .tx_queue_setup = idpf_tx_queue_setup,
> .dev_infos_get = idpf_dev_info_get,
> };
>
> @@ -54,11 +56,23 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> dev_info->min_mtu = RTE_ETHER_MIN_MTU;
>
> dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> - dev_info->dev_capa = 0;
> + dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
I've played with two Intel PMDs: i40e and ice and both have
bugs in runtime queue setup. That's why I'm trying to review
it carefully in a new driver. So, I'd like to repeat once
again: the feature does not make sense without device start
support since corresponding code cann't be in-place and
cannot be reviewed here.
> dev_info->rx_offload_capa = 0;
> -
Unrlated change
> dev_info->tx_offload_capa = 0;
>
> + dev_info->default_txconf = (struct rte_eth_txconf) {
> + .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
> + .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
> + .offloads = 0,
There is no necessity to initialize to zero explicitly since
the compiler does it for you implicitly.
> + };
> +
> + dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
> + .nb_max = IDPF_MAX_RING_DESC,
> + .nb_min = IDPF_MIN_RING_DESC,
> + .nb_align = IDPF_ALIGN_RING_DESC,
> + };
> +
> +
> return 0;
> }
>
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> new file mode 100644
> index 0000000000..df0ed772c1
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -0,0 +1,371 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <ethdev_driver.h>
> +#include <rte_net.h>
> +
> +#include "idpf_ethdev.h"
> +#include "idpf_rxtx.h"
> +
> +static inline int
There is no point to make it inline. It is a control path.
Please, keep it to the compiler to do its job.
> +check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
> + uint16_t tx_free_thresh)
[snip]
> +static inline void
> +reset_split_tx_descq(struct idpf_tx_queue *txq)
same here since it looks like control path as well.
[snip]
> +{
> + struct idpf_tx_entry *txe;
> + uint32_t i, size;
> + uint16_t prev;
> +
> + if (txq == NULL) {
> + PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
> + return;
> + }
> +
> + size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
> + for (i = 0; i < size; i++)
> + ((volatile char *)txq->desc_ring)[i] = 0;
Please, add a comment which explains why volatile is
required here.
> +
> + txe = txq->sw_ring;
> + prev = (uint16_t)(txq->sw_nb_desc - 1);
> + for (i = 0; i < txq->sw_nb_desc; i++) {
> + txe[i].mbuf = NULL;
> + txe[i].last_id = i;
> + txe[prev].next_id = i;
> + prev = i;
> + }
> +
> + txq->tx_tail = 0;
> + txq->nb_used = 0;
> +
> + /* Use this as next to clean for split desc queue */
> + txq->last_desc_cleaned = 0;
> + txq->sw_tail = 0;
> + txq->nb_free = txq->nb_tx_desc - 1;
> +}
> +
> +static inline void
> +reset_split_tx_complq(struct idpf_tx_queue *cq)
same here
> +{
> + uint32_t i, size;
> +
> + if (cq == NULL) {
> + PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
> + return;
> + }
> +
> + size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
> + for (i = 0; i < size; i++)
> + ((volatile char *)cq->compl_ring)[i] = 0;
same ehre
> +
> + cq->tx_tail = 0;
> + cq->expected_gen_id = 1;
> +}
> +
> +static inline void
> +reset_single_tx_queue(struct idpf_tx_queue *txq)
same here
> +{
> + struct idpf_tx_entry *txe;
> + uint32_t i, size;
> + uint16_t prev;
> +
> + if (txq == NULL) {
> + PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
> + return;
> + }
> +
> + txe = txq->sw_ring;
> + size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
> + for (i = 0; i < size; i++)
> + ((volatile char *)txq->tx_ring)[i] = 0;
same here
> +
> + prev = (uint16_t)(txq->nb_tx_desc - 1);
> + for (i = 0; i < txq->nb_tx_desc; i++) {
> + txq->tx_ring[i].qw1.cmd_dtype =
> + rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
> + txe[i].mbuf = NULL;
> + txe[i].last_id = i;
> + txe[prev].next_id = i;
> + prev = i;
> + }
> +
> + txq->tx_tail = 0;
> + txq->nb_used = 0;
> +
> + txq->last_desc_cleaned = txq->nb_tx_desc - 1;
> + txq->nb_free = txq->nb_tx_desc - 1;
> +
> + txq->next_dd = txq->rs_thresh - 1;
> + txq->next_rs = txq->rs_thresh - 1;
> +}
> +
> +static int
> +idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
> + uint16_t nb_desc, unsigned int socket_id,
> + const struct rte_eth_txconf *tx_conf)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t tx_rs_thresh, tx_free_thresh;
> + struct idpf_hw *hw = &adapter->hw;
> + struct idpf_tx_queue *txq, *cq;
> + const struct rte_memzone *mz;
> + uint32_t ring_size;
> + uint64_t offloads;
> +
> + offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
> +
> + if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
> + nb_desc > IDPF_MAX_RING_DESC ||
> + nb_desc < IDPF_MIN_RING_DESC) {
> + PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
> + nb_desc);
> + return -EINVAL;
> + }
You don't need to check it here, since ethdev does it for you
before driver operation invocation.
> +
> + tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
compare vs 0 explicitly
> + tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
> + tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
compare vs 0 explicitly
> + tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
> + if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
> + return -EINVAL;
> +
> + /* Allocate the TX queue data structure. */
> + txq = rte_zmalloc_socket("idpf split txq",
> + sizeof(struct idpf_tx_queue),
> + RTE_CACHE_LINE_SIZE,
> + socket_id);
> + if (txq == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
> + return -ENOMEM;
> + }
> +
> + txq->nb_tx_desc = nb_desc;
> + txq->rs_thresh = tx_rs_thresh;
> + txq->free_thresh = tx_free_thresh;
> + txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
> + txq->port_id = dev->data->port_id;
> + txq->offloads = offloads;
> + txq->tx_deferred_start = tx_conf->tx_deferred_start;
Deferred start is not supported at this point and
request with deferred start on should be rejected.
> +
> + /* Allocate software ring */
> + txq->sw_nb_desc = 2 * nb_desc;
> + txq->sw_ring =
> + rte_zmalloc_socket("idpf split tx sw ring",
> + sizeof(struct idpf_tx_entry) *
> + txq->sw_nb_desc,
> + RTE_CACHE_LINE_SIZE,
> + socket_id);
> + if (txq->sw_ring == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
> + rte_free(txq);
> + return -ENOMEM;
> + }
> +
> + /* Allocate TX hardware ring descriptors. */
> + ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
> + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
> + mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
> + ring_size, IDPF_RING_BASE_ALIGN,
> + socket_id);
> + if (mz == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
> + rte_free(txq->sw_ring);
> + rte_free(txq);
> + return -ENOMEM;
> + }
> + txq->tx_ring_phys_addr = mz->iova;
> + txq->desc_ring = (struct idpf_flex_tx_sched_desc *)mz->addr;
> +
> + txq->mz = mz;
> + reset_split_tx_descq(txq);
> + txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
> + queue_idx * vport->chunks_info.tx_qtail_spacing);
> +
> + /* Allocate the TX completion queue data structure. */
> + txq->complq = rte_zmalloc_socket("idpf splitq cq",
> + sizeof(struct idpf_tx_queue),
> + RTE_CACHE_LINE_SIZE,
> + socket_id);
> + cq = txq->complq;
> + if (cq == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
Free previously allocated resources including txq->complq.
However, I'd recomment to use goto style to do cleanup in
this case.
> + return -ENOMEM;
> + }
> + cq->nb_tx_desc = 2 * nb_desc;
> + cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
> + cq->port_id = dev->data->port_id;
> + cq->txqs = dev->data->tx_queues;
> + cq->tx_start_qid = vport->chunks_info.tx_start_qid;
> +
> + ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
> + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
> + mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
> + ring_size, IDPF_RING_BASE_ALIGN,
> + socket_id);
> + if (mz == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
> + rte_free(txq->sw_ring);
> + rte_free(txq);
Free txq->complq and txq->mz
> + return -ENOMEM;
> + }
> + cq->tx_ring_phys_addr = mz->iova;
> + cq->compl_ring = (struct idpf_splitq_tx_compl_desc *)mz->addr;
> + cq->mz = mz;
> + reset_split_tx_complq(cq);
> +
> + txq->q_set = true;
> + dev->data->tx_queues[queue_idx] = txq;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
> + uint16_t nb_desc, unsigned int socket_id,
> + const struct rte_eth_txconf *tx_conf)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t tx_rs_thresh, tx_free_thresh;
> + struct idpf_hw *hw = &adapter->hw;
> + const struct rte_memzone *mz;
> + struct idpf_tx_queue *txq;
> + uint32_t ring_size;
> + uint64_t offloads;
> +
> + offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
> +
> + if (nb_desc % IDPF_ALIGN_RING_DESC != 0 ||
> + nb_desc > IDPF_MAX_RING_DESC ||
> + nb_desc < IDPF_MIN_RING_DESC) {
> + PMD_INIT_LOG(ERR, "Number (%u) of transmit descriptors is invalid",
> + nb_desc);
> + return -EINVAL;
> + }
You don't need to check it here, since ethdev does it for you
before driver operation invocation.
> +
> + tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
> + tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
> + tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
> + tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
> + if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
> + return -EINVAL;
> +
> + /* Allocate the TX queue data structure. */
> + txq = rte_zmalloc_socket("idpf txq",
> + sizeof(struct idpf_tx_queue),
> + RTE_CACHE_LINE_SIZE,
> + socket_id);
> + if (txq == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
> + return -ENOMEM;
> + }
> +
> + /* TODO: vlan offload */
> +
> + txq->nb_tx_desc = nb_desc;
> + txq->rs_thresh = tx_rs_thresh;
> + txq->free_thresh = tx_free_thresh;
> + txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
> + txq->port_id = dev->data->port_id;
> + txq->offloads = offloads;
> + txq->tx_deferred_start = tx_conf->tx_deferred_start;
same here
> +
> + /* Allocate software ring */
> + txq->sw_ring =
> + rte_zmalloc_socket("idpf tx sw ring",
> + sizeof(struct idpf_tx_entry) * nb_desc,
> + RTE_CACHE_LINE_SIZE,
> + socket_id);
> + if (txq->sw_ring == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
> + rte_free(txq);
> + return -ENOMEM;
> + }
> +
> + /* Allocate TX hardware ring descriptors. */
> + ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
> + ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
> + mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
> + ring_size, IDPF_RING_BASE_ALIGN,
> + socket_id);
> + if (mz == NULL) {
> + PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
> + rte_free(txq->sw_ring);
> + rte_free(txq);
> + return -ENOMEM;
> + }
> +
> + txq->tx_ring_phys_addr = mz->iova;
> + txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
> +
> + txq->mz = mz;
> + reset_single_tx_queue(txq);
> + txq->q_set = true;
> + dev->data->tx_queues[queue_idx] = txq;
> + txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
> + queue_idx * vport->chunks_info.tx_qtail_spacing);
I'm sorry, but it looks like too much code is duplicated.
I guess it could be a shared function to avoid it.
> +
> + return 0;
> +}
> +
> +int
> +idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
> + uint16_t nb_desc, unsigned int socket_id,
> + const struct rte_eth_txconf *tx_conf)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
> + return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
> + socket_id, tx_conf);
> + else
> + return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
> + socket_id, tx_conf);
> +}
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 05/18] net/idpf: add support for device start and stop
2022-10-24 13:12 ` [PATCH v11 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-25 9:49 ` Andrew Rybchenko
2022-10-26 8:38 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 9:49 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/24/22 16:12, Junfeng Guo wrote:
> Add dev ops dev_start, dev_stop and link_update.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 89 ++++++++++++++++++++++++++++++++++
> drivers/net/idpf/idpf_ethdev.h | 5 ++
> 2 files changed, 94 insertions(+)
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index 1d2075f466..4c7a2d0748 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -29,17 +29,42 @@ static const char * const idpf_valid_args[] = {
> };
>
> static int idpf_dev_configure(struct rte_eth_dev *dev);
> +static int idpf_dev_start(struct rte_eth_dev *dev);
> +static int idpf_dev_stop(struct rte_eth_dev *dev);
> static int idpf_dev_close(struct rte_eth_dev *dev);
> static int idpf_dev_info_get(struct rte_eth_dev *dev,
> struct rte_eth_dev_info *dev_info);
> static void idpf_adapter_rel(struct idpf_adapter *adapter);
>
> +int
> +idpf_dev_link_update(struct rte_eth_dev *dev,
> + __rte_unused int wait_to_complete)
Why is it global? IMHO it should be static now and should be
made global later if you really need it.
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct rte_eth_link new_link;
> +
> + memset(&new_link, 0, sizeof(new_link));
> +
> + new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
> +
> + new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
> + new_link.link_status = vport->link_up ? RTE_ETH_LINK_UP :
> + RTE_ETH_LINK_DOWN;
> + new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
> + RTE_ETH_LINK_SPEED_FIXED);
> +
> + return rte_eth_linkstatus_set(dev, &new_link);
> +}
> +
> static const struct eth_dev_ops idpf_eth_dev_ops = {
> .dev_configure = idpf_dev_configure,
> + .dev_start = idpf_dev_start,
> + .dev_stop = idpf_dev_stop,
> .dev_close = idpf_dev_close,
> .rx_queue_setup = idpf_rx_queue_setup,
> .tx_queue_setup = idpf_tx_queue_setup,
> .dev_infos_get = idpf_dev_info_get,
> + .link_update = idpf_dev_link_update,
> };
>
> static int
> @@ -233,6 +258,70 @@ idpf_dev_configure(struct rte_eth_dev *dev)
> return 0;
> }
>
> +static int
> +idpf_start_queues(struct rte_eth_dev *dev)
> +{
> + struct idpf_rx_queue *rxq;
> + struct idpf_tx_queue *txq;
> + int err = 0;
> + int i;
> +
> + for (i = 0; i < dev->data->nb_tx_queues; i++) {
> + txq = dev->data->tx_queues[i];
> + if (txq == NULL || txq->tx_deferred_start)
> + continue;
> +
> + PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
> + return -ENOTSUP;
> + }
> +
> + for (i = 0; i < dev->data->nb_rx_queues; i++) {
> + rxq = dev->data->rx_queues[i];
> + if (rxq == NULL || rxq->rx_deferred_start)
> + continue;
> +
> + PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
> + return -ENOTSUP;
> + }
> +
> + return err;
> +}
> +
> +static int
> +idpf_dev_start(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + if (dev->data->mtu > vport->max_mtu) {
> + PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
> + return -1;
> + }
> +
> + vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
> +
> + if (idpf_start_queues(dev) != 0) {
> + PMD_DRV_LOG(ERR, "Failed to start queues");
> + return -1;
> + }
> +
> + if (idpf_vc_ena_dis_vport(vport, true) != 0) {
> + PMD_DRV_LOG(ERR, "Failed to enable vport");
Don't you need to stop queues here?
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_stop(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
Stop queues?
> +
> + idpf_vc_ena_dis_vport(vport, false);
> +
> + return 0;
> +}
> +
> static int
> idpf_dev_close(struct rte_eth_dev *dev)
> {
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> index c0ae801fd5..070531cc48 100644
> --- a/drivers/net/idpf/idpf_ethdev.h
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -105,6 +105,9 @@ struct idpf_vport {
> /* Chunk info */
> struct idpf_chunks_info chunks_info;
>
> + /* Event from ipf */
> + bool link_up;
> +
It is a dead code. Since it is read, but never written in the
patch.
> uint16_t devarg_id;
> };
>
> @@ -195,6 +198,8 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
> }
>
> struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
> +int idpf_dev_link_update(struct rte_eth_dev *dev,
> + __rte_unused int wait_to_complete);
> void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
> int idpf_vc_check_api_version(struct idpf_adapter *adapter);
> int idpf_vc_get_caps(struct idpf_adapter *adapter);
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 09/18] net/idpf: add support for packet type get
2022-10-24 13:12 ` [PATCH v11 09/18] net/idpf: add support for packet type get Junfeng Guo
@ 2022-10-25 9:57 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 9:57 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Wenjun Wu
On 10/24/22 16:12, Junfeng Guo wrote:
> Add dev ops dev_supported_ptypes_get.
>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
Again, the patch should go after datapath implementation to
make it reviewable.
[snip]
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> index 30e1c0831e..a03068df85 100644
> --- a/doc/guides/nics/features/idpf.ini
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -8,4 +8,5 @@
> ;
> [Features]
> Queue start/stop = Y
> +Packet type parsing = Y
It is false without datapath implementation.
> Linux = Y
[snip]
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> index 298eaf0a1a..c3c4acb69f 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
> @@ -8,6 +8,25 @@
> #include "idpf_ethdev.h"
> #include "idpf_rxtx.h"
>
> +const uint32_t *
> +idpf_dev_supported_ptypes_get(struct rte_eth_dev *dev __rte_unused)
> +{
> + static const uint32_t ptypes[] = {
> + RTE_PTYPE_L2_ETHER,
> + RTE_PTYPE_L3_IPV4_EXT_UNKNOWN,
> + RTE_PTYPE_L3_IPV6_EXT_UNKNOWN,
> + RTE_PTYPE_L4_FRAG,
> + RTE_PTYPE_L4_NONFRAG,
> + RTE_PTYPE_L4_UDP,
> + RTE_PTYPE_L4_TCP,
> + RTE_PTYPE_L4_SCTP,
> + RTE_PTYPE_L4_ICMP,
> + RTE_PTYPE_UNKNOWN
> + };
It looks like above array must not be static. It should be
dynamically built based on idpf_get_pkt_type().
> +
> + return ptypes;
> +}
> +
> static inline int
> check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
> {
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 15/18] net/idpf: add support for Rx offloading
2022-10-24 13:12 ` [PATCH v11 15/18] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-25 10:03 ` Andrew Rybchenko
2022-10-28 1:48 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 10:03 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/24/22 16:12, Junfeng Guo wrote:
> Add Rx offloading support:
> - support CHKSUM and RSS offload for split queue model
> - support CHKSUM offload for single queue model
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> doc/guides/nics/features/idpf.ini | 2 +
> drivers/net/idpf/idpf_ethdev.c | 9 ++-
> drivers/net/idpf/idpf_rxtx.c | 122 ++++++++++++++++++++++++++++++
> 3 files changed, 132 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
> index d4eb9b374c..c86d9378ea 100644
> --- a/doc/guides/nics/features/idpf.ini
> +++ b/doc/guides/nics/features/idpf.ini
> @@ -9,5 +9,7 @@
> [Features]
> Queue start/stop = Y
> MTU update = Y
> +L3 checksum offload = P
> +L4 checksum offload = P
RSS hash missing
> Packet type parsing = Y
> Linux = Y
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> index 739cf31d65..d8cc423a23 100644
> --- a/drivers/net/idpf/idpf_ethdev.c
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -94,7 +94,14 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> dev_info->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
> - dev_info->rx_offload_capa = 0;
> +
> + dev_info->rx_offload_capa =
> + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
> + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
> + RTE_ETH_RX_OFFLOAD_RSS_HASH;
As I understand you know mode here and you should not report
offload which is not supported in current mode (RSS vs single
queue).
> +
> dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
>
> dev_info->default_rxconf = (struct rte_eth_rxconf) {
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 12/18] net/idpf: add support for basic Tx datapath
2022-10-24 13:12 ` [PATCH v11 12/18] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-25 10:12 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 10:12 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/24/22 16:12, Junfeng Guo wrote:
> Add basic Tx support in split queue mode and single queue mode.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> +/* TX prep functions */
> +uint16_t
> +idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
> + uint16_t nb_pkts)
> +{
> + int i, ret;
> + struct rte_mbuf *m;
> +
> + for (i = 0; i < nb_pkts; i++) {
> + m = tx_pkts[i];
> +
> + if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
> + rte_errno = EINVAL;
> + return i;
> + }
> +
> + ret = rte_net_intel_cksum_prepare(m);
I'm wondering what it does here if you don't support Tx
checksum offloads yet.
> + if (ret != 0) {
> + rte_errno = -ret;
> + return i;
> + }
> + }
> +
> + return i;
> +}
> +
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 16/18] net/idpf: add support for Tx offloading
2022-10-24 13:12 ` [PATCH v11 16/18] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-25 10:14 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-25 10:14 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/24/22 16:12, Junfeng Guo wrote:
> Add Tx offloading support:
> - support TSO
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
> index 143c8b69f3..8f82cf1b59 100644
> --- a/drivers/net/idpf/idpf_rxtx.c
> +++ b/drivers/net/idpf/idpf_rxtx.c
[snip]
> @@ -1968,16 +2070,44 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts)
> {
> int i, ret;
> + uint64_t ol_flags;
> struct rte_mbuf *m;
>
> for (i = 0; i < nb_pkts; i++) {
> m = tx_pkts[i];
> + ol_flags = m->ol_flags;
> +
> + /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
> + if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
> + if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
> + rte_errno = EINVAL;
> + return i;
> + }
It is non-TSO branch and it should be here before the patch.
> + } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
> + (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
> + (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
> + /* MSS outside the range are considered malicious */
> + rte_errno = EINVAL;
> + return i;
> + }
> +
> + if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
> + rte_errno = ENOTSUP;
> + return i;
> + }
>
> if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
> rte_errno = EINVAL;
> return i;
> }
>
> +#ifdef RTE_LIBRTE_ETHDEV_DEBUG
> + ret = rte_validate_tx_offload(m);
> + if (ret != 0) {
> + rte_errno = -ret;
> + return i;
> + }
> +#endif
> ret = rte_net_intel_cksum_prepare(m);
> if (ret != 0) {
> rte_errno = -ret;
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v11 02/18] net/idpf: add support for device initialization
2022-10-25 8:57 ` Andrew Rybchenko
@ 2022-10-26 8:28 ` Xing, Beilei
2022-10-28 15:14 ` Andrew Rybchenko
1 sibling, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-26 8:28 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Tuesday, October 25, 2022 4:57 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v11 02/18] net/idpf: add support for device initialization
>
> On 10/24/22 16:12, Junfeng Guo wrote:
> > Support device init and add the following dev ops:
> > - dev_configure
> > - dev_close
> > - dev_infos_get
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > diff --git a/doc/guides/nics/features/idpf.ini
> > b/doc/guides/nics/features/idpf.ini
> > new file mode 100644
> > index 0000000000..7a44b8b5e4
> > --- /dev/null
> > +++ b/doc/guides/nics/features/idpf.ini
> > @@ -0,0 +1,10 @@
> > +;
> > +; Supported features of the 'idpf' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +; A feature with "P" indicates only be supported when non-vector path
> > +; is selected.
>
> The statement should be added when the first P appears.
Thanks for the comments, all the comments are addressed in next version except the below one.
[snip]
> > +};
> > +
> > +struct idpf_vport {
> > + struct idpf_adapter *adapter; /* Backreference to associated adapter
> */
> > + uint16_t vport_id;
> > + uint32_t txq_model;
> > + uint32_t rxq_model;
> > + uint16_t num_tx_q;
>
> Shouldn't it be set as the result of configure?
> data->nb_tx_queues which is the result of the configure is not
> used in the patch?
The num_tx_q here is different from data->nb_tx_queues.
The idpf PMD requires fixed txqs and rxqs when the idpf PMD requires creating a vport.
The num_tx_q and the num_rxq_q are the values returned by backend.
But there'll be only data->nb_tx_queues txqs configured during dev_start.
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v11 03/18] net/idpf: add Tx queue setup
2022-10-25 9:40 ` Andrew Rybchenko
@ 2022-10-26 8:34 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-26 8:34 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Tuesday, October 25, 2022 5:40 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v11 03/18] net/idpf: add Tx queue setup
>
> On 10/24/22 16:12, Junfeng Guo wrote:
> > Add support for tx_queue_setup ops.
> >
> > In the single queue model, the same descriptor queue is used by SW to
> > post buffer descriptors to HW and by HW to post completed descriptors
> > to SW.
> >
> > In the split queue model, "RX buffer queues" are used to pass
> > descriptor buffers from SW to HW while Rx queues are used only to pass
> > the descriptor completions, that is, descriptors that point to
> > completed buffers, from HW to SW. This is contrary to the single queue
> > model in which Rx queues are used for both purposes.
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> > +
> > + size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
> > + for (i = 0; i < size; i++)
> > + ((volatile char *)txq->desc_ring)[i] = 0;
>
> Please, add a comment which explains why volatile is required here.
Volatile is used here to make sure the memory write through when touch desc.
It follows Intel PMD's coding style.
>
> > +
> > + txq->tx_ring_phys_addr = mz->iova;
> > + txq->tx_ring = (struct idpf_flex_tx_desc *)mz->addr;
> > +
> > + txq->mz = mz;
> > + reset_single_tx_queue(txq);
> > + txq->q_set = true;
> > + dev->data->tx_queues[queue_idx] = txq;
> > + txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
> > + queue_idx * vport->chunks_info.tx_qtail_spacing);
>
> I'm sorry, but it looks like too much code is duplicated.
> I guess it could be a shared function to avoid it.
It makes sense to optimize the queue setup, but can we improve it in next release
due to the RC2 deadline?
And this part should be common module shared by idpf PMD and a new PMD,
It should be in driver/common/idpf folder in DPDK 23.03.
>
> > +
> > + return 0;
> > +}
> > +
> > +int
> > +idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
> > + uint16_t nb_desc, unsigned int socket_id,
> > + const struct rte_eth_txconf *tx_conf) {
> > + struct idpf_vport *vport = dev->data->dev_private;
> > +
> > + if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
> > + return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
> > + socket_id, tx_conf);
> > + else
> > + return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
> > + socket_id, tx_conf);
> > +}
>
> [snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v11 05/18] net/idpf: add support for device start and stop
2022-10-25 9:49 ` Andrew Rybchenko
@ 2022-10-26 8:38 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-26 8:38 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Tuesday, October 25, 2022 5:50 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v11 05/18] net/idpf: add support for device start and
> stop
>
> On 10/24/22 16:12, Junfeng Guo wrote:
> > Add dev ops dev_start, dev_stop and link_update.
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> > ---
> > drivers/net/idpf/idpf_ethdev.c | 89
> ++++++++++++++++++++++++++++++++++
> > drivers/net/idpf/idpf_ethdev.h | 5 ++
> > 2 files changed, 94 insertions(+)
> >
> > diff --git a/drivers/net/idpf/idpf_ethdev.c
> > b/drivers/net/idpf/idpf_ethdev.c index 1d2075f466..4c7a2d0748 100644
> > --- a/drivers/net/idpf/idpf_ethdev.c
> > +++ b/drivers/net/idpf/idpf_ethdev.c
> > @@ -29,17 +29,42 @@ static const char * const idpf_valid_args[] = {
> > };
> > +static int
> > +idpf_start_queues(struct rte_eth_dev *dev) {
> > + struct idpf_rx_queue *rxq;
> > + struct idpf_tx_queue *txq;
> > + int err = 0;
> > + int i;
> > +
> > + for (i = 0; i < dev->data->nb_tx_queues; i++) {
> > + txq = dev->data->tx_queues[i];
> > + if (txq == NULL || txq->tx_deferred_start)
> > + continue;
> > +
> > + PMD_DRV_LOG(ERR, "Start Tx queues not supported yet");
> > + return -ENOTSUP;
> > + }
> > +
> > + for (i = 0; i < dev->data->nb_rx_queues; i++) {
> > + rxq = dev->data->rx_queues[i];
> > + if (rxq == NULL || rxq->rx_deferred_start)
> > + continue;
> > +
> > + PMD_DRV_LOG(ERR, "Start Rx queues not supported yet");
> > + return -ENOTSUP;
> > + }
> > +
> > + return err;
> > +}
> > +
> > +static int
> > +idpf_dev_start(struct rte_eth_dev *dev) {
> > + struct idpf_vport *vport = dev->data->dev_private;
> > +
> > + if (dev->data->mtu > vport->max_mtu) {
> > + PMD_DRV_LOG(ERR, "MTU should be less than %d", vport-
> >max_mtu);
> > + return -1;
> > + }
> > +
> > + vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
> > +
> > + if (idpf_start_queues(dev) != 0) {
> > + PMD_DRV_LOG(ERR, "Failed to start queues");
> > + return -1;
> > + }
> > +
> > + if (idpf_vc_ena_dis_vport(vport, true) != 0) {
> > + PMD_DRV_LOG(ERR, "Failed to enable vport");
>
> Don't you need to stop queues here?
In this patch, we didn't implement start HW queues, so I will remove start queues here
And add start_queues/stop_queues once the APIs are finished.
>
> > + return -1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > +idpf_dev_stop(struct rte_eth_dev *dev) {
> > + struct idpf_vport *vport = dev->data->dev_private;
>
> Stop queues?
Same.
>
> > +
> > + idpf_vc_ena_dis_vport(vport, false);
> > +
> > + return 0;
> > +}
> > +
> > static int
> > idpf_dev_close(struct rte_eth_dev *dev)
> > {
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 00/18] add support for idpf PMD in DPDK
2022-10-24 13:12 ` [PATCH v11 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 01/18] common/idpf: introduce common library Junfeng Guo
` (17 more replies)
0 siblings, 18 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style accoding to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support packet type get
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 18 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1280 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2305 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1430 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15254 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 01/18] common/idpf: introduce common library
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 02/18] net/idpf: add support for device initialization Junfeng Guo
` (16 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8502 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 02/18] net/idpf: add support for device initialization
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 03/18] net/idpf: add Tx queue setup Junfeng Guo
` (15 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 886 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 468 +++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1709 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..4af790de37 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -160,6 +160,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..112dccf5c9
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,886 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -1;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -1;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -1;
+ }
+
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -1;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ return adapter;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..9392711b61
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,468 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -1;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -1;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 03/18] net/idpf: add Tx queue setup
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 04/18] net/idpf: add Rx " Junfeng Guo
` (14 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/idpf_vchnl.c | 2 -
drivers/net/idpf/meson.build | 1 +
5 files changed, 448 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 112dccf5c9..11f8b4ba1c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -37,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -56,6 +58,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9392711b61..4dae1a0b36 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -22,8 +22,6 @@
#include "idpf_ethdev.h"
-#define IDPF_CTLQ_LEN 64
-
static int
idpf_vc_clean(struct idpf_adapter *adapter)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 04/18] net/idpf: add Rx queue setup
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 03/18] net/idpf: add Tx queue setup Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 05/18] net/idpf: add support for device start and stop Junfeng Guo
` (13 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 11f8b4ba1c..0585153f69 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -38,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -63,12 +64,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 05/18] net/idpf: add support for device start and stop
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 04/18] net/idpf: add Rx " Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 06/18] net/idpf: add support for queue start Junfeng Guo
` (12 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 51 ++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0585153f69..e1db7ccb4f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,17 +30,38 @@ static const char * const idpf_valid_args[] = {
};
static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
@@ -284,6 +305,36 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -1;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ if (idpf_vc_ena_dis_vport(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 06/18] net/idpf: add support for queue start
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 07/18] net/idpf: add support for queue stop Junfeng Guo
` (11 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 40 +++
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 218 ++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 720 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index e1db7ccb4f..29bac29bc3 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -58,6 +58,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -305,6 +307,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -317,6 +352,11 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ if (idpf_start_queues(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return -1;
+ }
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
return -1;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..8f73e3f3bb 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -349,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -480,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -585,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -709,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -762,3 +766,217 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4dae1a0b36..11d915cf4e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -440,6 +445,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 07/18] net/idpf: add support for queue stop
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 06/18] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 08/18] net/idpf: add queue release Junfeng Guo
` (10 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++-
drivers/net/idpf/idpf_rxtx.c | 148 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 ++++++++++++++
5 files changed, 242 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..c25fa5de3f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+Queue start/stop = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 29bac29bc3..05f087a03c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,7 +59,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -347,22 +349,26 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -1;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return -1;
+ goto err_mtu;
}
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- return -1;
+ goto err_vport;
}
return 0;
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return -1;
}
static int
@@ -372,6 +378,8 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
+ idpf_stop_queues(dev);
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8f73e3f3bb..ef11855978 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -980,3 +1041,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
+
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 11d915cf4e..1ba59929a0 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -887,6 +887,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 08/18] net/idpf: add queue release
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 07/18] net/idpf: add support for queue stop Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 09/18] net/idpf: add support for MTU configuration Junfeng Guo
` (9 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 05f087a03c..dee1e03d43 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -63,7 +63,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ef11855978..382f0090f3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 09/18] net/idpf: add support for MTU configuration
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 08/18] net/idpf: add queue release Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
` (8 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++++++++++++++
2 files changed, 15 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c25fa5de3f..c8f4a0d6ed 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -5,6 +5,7 @@
;
[Features]
Queue start/stop = Y
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index dee1e03d43..94f4671057 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
static int
@@ -68,6 +69,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -110,6 +112,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 10/18] net/idpf: add support for basic Rx datapath
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 09/18] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
` (7 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 94f4671057..df2a760673 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -375,6 +375,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 382f0090f3..4e3fa39b9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,3 +1209,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 11/18] net/idpf: add support for basic Tx datapath
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 12/18] net/idpf: support packet type get Junfeng Guo
` (6 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index df2a760673..6fb56e584d 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -88,6 +88,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -376,6 +378,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4e3fa39b9d..86942b2b5f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1366,6 +1366,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1472,6 +1614,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1482,3 +1826,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 12/18] net/idpf: support packet type get
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (5 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Get packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6fb56e584d..630bdabcd4 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -709,6 +709,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 86942b2b5f..f9f751a6ad 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1282,6 +1282,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1301,6 +1302,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1348,6 +1350,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1534,6 +1540,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1552,6 +1559,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1604,6 +1612,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 1ba59929a0..ac399d331a 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -980,3 +1194,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 13/18] net/idpf: add support for write back based on ITR expire
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 12/18] net/idpf: support packet type get Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 14/18] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 111 +++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 630bdabcd4..8045073780 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -327,6 +327,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps != 0) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -364,6 +448,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
@@ -372,6 +460,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -401,6 +506,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -412,6 +521,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -766,6 +880,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ac399d331a..dc629be741 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -1020,6 +1024,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 14/18] net/idpf: add support for RSS
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 15/18] net/idpf: add support for Rx offloading Junfeng Guo
` (3 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 26 ++++++++
drivers/net/idpf/idpf_vchnl.c | 97 +++++++++++++++++++++++++++
3 files changed, 239 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 8045073780..ea7d3bfd7e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
@@ -198,6 +199,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -218,6 +221,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -275,10 +282,102 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
@@ -324,6 +423,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -1;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -521,6 +631,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index dc629be741..c354a5815f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -663,6 +666,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 15/18] net/idpf: add support for Rx offloading
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (13 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 14/18] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 16/18] net/idpf: add support for Tx offloading Junfeng Guo
` (2 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 122 ++++++++++++++++++++++++++++++
3 files changed, 133 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c8f4a0d6ed..6eefac9529 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,9 +3,14 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
Queue start/stop = Y
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ea7d3bfd7e..88f3db3267 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -89,6 +89,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f9f751a6ad..cdd20702d8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,72 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
@@ -1283,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1350,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1514,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1547,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1612,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 16/18] net/idpf: add support for Tx offloading
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (14 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 15/18] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Tx offloading support:
- support TSO
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 125 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 ++++++
4 files changed, 149 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 6eefac9529..86c182021f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,6 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 88f3db3267..868f28d003 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -95,7 +95,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cdd20702d8..1403e4bafe 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1938,6 +2040,17 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
@@ -1946,6 +2059,14 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
+
return i;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (15 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 16/18] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 868f28d003..62b13b602e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -97,7 +97,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 1403e4bafe..bdb67ceab8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2070,25 +2074,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v12 18/18] net/idpf: add support for timestamp offload
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (16 preceding siblings ...)
2022-10-26 10:10 ` [PATCH v12 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-26 10:10 ` Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-26 10:10 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 86c182021f..5fbad9600c 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -12,6 +12,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 62b13b602e..cd18d59fb7 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -98,7 +100,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
- RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index bdb67ceab8..0fc4dca9ec 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 00/18] add support for idpf PMD in DPDK
2022-10-26 10:10 ` [PATCH v12 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 01/18] common/idpf: introduce common library Junfeng Guo
` (17 more replies)
0 siblings, 18 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style accoding to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 18 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1280 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1430 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15257 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 01/18] common/idpf: introduce common library
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 02/18] net/idpf: add support for device initialization Junfeng Guo
` (16 subsequent siblings)
17 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8502 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 02/18] net/idpf: add support for device initialization
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 03/18] net/idpf: add Tx queue setup Junfeng Guo
` (15 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 886 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 468 +++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1709 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..4af790de37 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -160,6 +160,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..112dccf5c9
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,886 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -1;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -1;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -1;
+ }
+
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -1;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ return adapter;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..9392711b61
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,468 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -1;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -1;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 03/18] net/idpf: add Tx queue setup
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 04/18] net/idpf: add Rx " Junfeng Guo
` (14 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/idpf_vchnl.c | 2 -
drivers/net/idpf/meson.build | 1 +
5 files changed, 448 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 112dccf5c9..11f8b4ba1c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -37,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -56,6 +58,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9392711b61..4dae1a0b36 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -22,8 +22,6 @@
#include "idpf_ethdev.h"
-#define IDPF_CTLQ_LEN 64
-
static int
idpf_vc_clean(struct idpf_adapter *adapter)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 04/18] net/idpf: add Rx queue setup
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 03/18] net/idpf: add Tx queue setup Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 05/18] net/idpf: add support for device start and stop Junfeng Guo
` (13 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 11f8b4ba1c..0585153f69 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -38,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -63,12 +64,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 05/18] net/idpf: add support for device start and stop
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 04/18] net/idpf: add Rx " Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 06/18] net/idpf: add support for queue start Junfeng Guo
` (12 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0585153f69..3430d00e92 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,17 +30,38 @@ static const char * const idpf_valid_args[] = {
};
static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
@@ -284,6 +305,40 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -1;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ if (idpf_vc_ena_dis_vport(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 06/18] net/idpf: add support for queue start
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 07/18] net/idpf: add support for queue stop Junfeng Guo
` (11 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 40 ++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 718 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 3430d00e92..abbf519977 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -58,6 +58,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -305,6 +307,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -317,7 +352,10 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ if (idpf_start_queues(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return -1;
+ }
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4dae1a0b36..11d915cf4e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -440,6 +445,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 07/18] net/idpf: add support for queue stop
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 06/18] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 08/18] net/idpf: add queue release Junfeng Guo
` (10 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++-
drivers/net/idpf/idpf_rxtx.c | 148 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 ++++++++++++++
5 files changed, 241 insertions(+), 4 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..c25fa5de3f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+Queue start/stop = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index abbf519977..05f087a03c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,7 +59,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -347,22 +349,26 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -1;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return -1;
+ goto err_mtu;
}
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- return -1;
+ goto err_vport;
}
return 0;
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return -1;
}
static int
@@ -372,7 +378,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 11d915cf4e..1ba59929a0 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -887,6 +887,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 08/18] net/idpf: add queue release
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 07/18] net/idpf: add support for queue stop Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 09/18] net/idpf: add support for MTU configuration Junfeng Guo
` (9 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 05f087a03c..dee1e03d43 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -63,7 +63,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 09/18] net/idpf: add support for MTU configuration
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 08/18] net/idpf: add queue release Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
` (8 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++++++++++++++
2 files changed, 15 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c25fa5de3f..c8f4a0d6ed 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -5,6 +5,7 @@
;
[Features]
Queue start/stop = Y
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index dee1e03d43..94f4671057 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
static int
@@ -68,6 +69,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -110,6 +112,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 10/18] net/idpf: add support for basic Rx datapath
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 09/18] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
` (7 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 94f4671057..df2a760673 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -375,6 +375,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 11/18] net/idpf: add support for basic Tx datapath
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 12/18] net/idpf: support parsing packet type Junfeng Guo
` (6 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index df2a760673..6fb56e584d 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -88,6 +88,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -376,6 +378,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 12/18] net/idpf: support parsing packet type
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-27 5:44 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (5 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:44 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6fb56e584d..630bdabcd4 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -709,6 +709,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 1ba59929a0..ac399d331a 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -980,3 +1194,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 13/18] net/idpf: add support for write back based on ITR expire
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-27 5:44 ` [PATCH v13 12/18] net/idpf: support parsing packet type Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 14/18] net/idpf: add support for RSS Junfeng Guo
` (4 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 111 +++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 630bdabcd4..8045073780 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -327,6 +327,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps != 0) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -364,6 +448,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
@@ -372,6 +460,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -401,6 +506,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -412,6 +521,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -766,6 +880,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ac399d331a..dc629be741 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -1020,6 +1024,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 14/18] net/idpf: add support for RSS
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-27 5:45 ` [PATCH v13 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 15/18] net/idpf: add support for Rx offloading Junfeng Guo
` (3 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 26 ++++++++
drivers/net/idpf/idpf_vchnl.c | 97 +++++++++++++++++++++++++++
3 files changed, 239 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 8045073780..ea7d3bfd7e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
@@ -198,6 +199,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -218,6 +221,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -275,10 +282,102 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
@@ -324,6 +423,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -1;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -521,6 +631,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index dc629be741..c354a5815f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -663,6 +666,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 15/18] net/idpf: add support for Rx offloading
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (13 preceding siblings ...)
2022-10-27 5:45 ` [PATCH v13 14/18] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 16/18] net/idpf: add support for Tx offloading Junfeng Guo
` (2 subsequent siblings)
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
3 files changed, 134 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c8f4a0d6ed..6eefac9529 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,9 +3,14 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
Queue start/stop = Y
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ea7d3bfd7e..88f3db3267 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -89,6 +89,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 16/18] net/idpf: add support for Tx offloading
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (14 preceding siblings ...)
2022-10-27 5:45 ` [PATCH v13 15/18] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Tx offloading support:
- support TSO
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 6eefac9529..86c182021f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,6 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 88f3db3267..868f28d003 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -95,7 +95,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..6b0ff3b341 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,6 +2043,17 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
@@ -1946,6 +2062,14 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
+
return i;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (15 preceding siblings ...)
2022-10-27 5:45 ` [PATCH v13 16/18] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 18/18] net/idpf: add support for timestamp offload Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 868f28d003..62b13b602e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -97,7 +97,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6b0ff3b341..3cad9b48e8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v13 18/18] net/idpf: add support for timestamp offload
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (16 preceding siblings ...)
2022-10-27 5:45 ` [PATCH v13 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-27 5:45 ` Junfeng Guo
17 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 5:45 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 86c182021f..5fbad9600c 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -12,6 +12,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 62b13b602e..84e7421d76 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -93,7 +95,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3cad9b48e8..c0e7fcb7e8 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 00/18] add support for idpf PMD in DPDK
2022-10-27 5:44 ` [PATCH v13 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 01/18] common/idpf: introduce common library Junfeng Guo
` (18 more replies)
0 siblings, 19 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
This patchset introduced the idpf (Infrastructure Data Path Function)
PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real
workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style accoding to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
v14:
- fix wrong position for rte_validate_tx_offload
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 18 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1280 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1430 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15257 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 01/18] common/idpf: introduce common library
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 02/18] net/idpf: add support for device initialization Junfeng Guo
` (17 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiao Wang
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8502 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 02/18] net/idpf: add support for device initialization
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 01/18] common/idpf: introduce common library Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-28 15:35 ` Andrew Rybchenko
2022-10-27 7:47 ` [PATCH v14 03/18] net/idpf: add Tx queue setup Junfeng Guo
` (16 subsequent siblings)
18 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 886 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 468 +++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1709 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 92b381bc30..34f8b9cc61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -764,6 +764,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 32c7544968..b4dd5ea522 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -33,6 +33,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 1c3daf141d..4af790de37 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -160,6 +160,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..112dccf5c9
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,886 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_close(struct rte_eth_dev *dev);
+static int idpf_dev_info_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info);
+static void idpf_adapter_rel(struct idpf_adapter *adapter);
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -1;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -1;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -1;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -1;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -1;
+ }
+
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -1;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -1;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -1;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -1;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -1;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -1;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ goto err_mbx;
+ }
+
+ if (idpf_vc_check_api_version(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ goto err_api;
+ }
+
+ if (idpf_vc_get_caps(adapter) != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return -1;
+}
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ return adapter;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ return NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -1;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_PROBE_AGAIN,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..9392711b61
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,468 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_CTLQ_LEN 64
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -1;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -1;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -1;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
+ caps_msg.seg_caps =
+ VIRTCHNL2_CAP_SEG_IPV4_TCP |
+ VIRTCHNL2_CAP_SEG_IPV4_UDP |
+ VIRTCHNL2_CAP_SEG_IPV4_SCTP |
+ VIRTCHNL2_CAP_SEG_IPV6_TCP |
+ VIRTCHNL2_CAP_SEG_IPV6_UDP |
+ VIRTCHNL2_CAP_SEG_IPV6_SCTP |
+ VIRTCHNL2_CAP_SEG_GENERIC;
+
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
+ caps_msg.other_caps =
+ VIRTCHNL2_CAP_SPLITQ_QSCHED |
+ VIRTCHNL2_CAP_CRC |
+ VIRTCHNL2_CAP_WB_ON_ITR |
+ VIRTCHNL2_CAP_PROMISC |
+ VIRTCHNL2_CAP_LINK_SPEED |
+ VIRTCHNL2_CAP_VLAN;
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -1;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (u8 *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 35bfa78dee..4a951b95f2 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -28,6 +28,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 03/18] net/idpf: add Tx queue setup
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 04/18] net/idpf: add Rx " Junfeng Guo
` (15 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/idpf_vchnl.c | 2 -
drivers/net/idpf/meson.build | 1 +
5 files changed, 448 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 112dccf5c9..11f8b4ba1c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -37,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -56,6 +58,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9392711b61..4dae1a0b36 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -22,8 +22,6 @@
#include "idpf_ethdev.h"
-#define IDPF_CTLQ_LEN 64
-
static int
idpf_vc_clean(struct idpf_adapter *adapter)
{
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 04/18] net/idpf: add Rx queue setup
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (2 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 03/18] net/idpf: add Tx queue setup Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 05/18] net/idpf: add support for device start and stop Junfeng Guo
` (14 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 11f8b4ba1c..0585153f69 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -38,6 +38,7 @@ static void idpf_adapter_rel(struct idpf_adapter *adapter);
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
@@ -63,12 +64,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 05/18] net/idpf: add support for device start and stop
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (3 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 04/18] net/idpf: add Rx " Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-28 15:45 ` Andrew Rybchenko
2022-10-27 7:47 ` [PATCH v14 06/18] net/idpf: add support for queue start Junfeng Guo
` (13 subsequent siblings)
18 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0585153f69..3430d00e92 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -30,17 +30,38 @@ static const char * const idpf_valid_args[] = {
};
static int idpf_dev_configure(struct rte_eth_dev *dev);
+static int idpf_dev_start(struct rte_eth_dev *dev);
+static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .link_update = idpf_dev_link_update,
};
static int
@@ -284,6 +305,40 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -1;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ if (idpf_vc_ena_dis_vport(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 06/18] net/idpf: add support for queue start
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (4 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-28 15:50 ` Andrew Rybchenko
2022-10-27 7:47 ` [PATCH v14 07/18] net/idpf: add support for queue stop Junfeng Guo
` (12 subsequent siblings)
18 siblings, 1 reply; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 40 ++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 718 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 3430d00e92..abbf519977 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -58,6 +58,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -305,6 +307,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -317,7 +352,10 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ if (idpf_start_queues(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return -1;
+ }
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4dae1a0b36..11d915cf4e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -440,6 +445,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 07/18] net/idpf: add support for queue stop
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (5 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 06/18] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 08/18] net/idpf: add queue release Junfeng Guo
` (11 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++-
drivers/net/idpf/idpf_rxtx.c | 148 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 ++++++++++++++
5 files changed, 241 insertions(+), 4 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..c25fa5de3f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+Queue start/stop = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index abbf519977..05f087a03c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,7 +59,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_stop = idpf_dev_stop,
.dev_close = idpf_dev_close,
.rx_queue_start = idpf_rx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_start = idpf_tx_queue_start,
+ .tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
@@ -347,22 +349,26 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -1;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return -1;
+ goto err_mtu;
}
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- return -1;
+ goto err_vport;
}
return 0;
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return -1;
}
static int
@@ -372,7 +378,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 11d915cf4e..1ba59929a0 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -887,6 +887,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 08/18] net/idpf: add queue release
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (6 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 07/18] net/idpf: add support for queue stop Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 09/18] net/idpf: add support for MTU configuration Junfeng Guo
` (10 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 05f087a03c..dee1e03d43 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -63,7 +63,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_setup = idpf_rx_queue_setup,
+ .rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_setup = idpf_tx_queue_setup,
+ .tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 09/18] net/idpf: add support for MTU configuration
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (7 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 08/18] net/idpf: add queue release Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
` (9 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 14 ++++++++++++++
2 files changed, 15 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c25fa5de3f..c8f4a0d6ed 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -5,6 +5,7 @@
;
[Features]
Queue start/stop = Y
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index dee1e03d43..94f4671057 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -35,6 +35,7 @@ static int idpf_dev_stop(struct rte_eth_dev *dev);
static int idpf_dev_close(struct rte_eth_dev *dev);
static int idpf_dev_info_get(struct rte_eth_dev *dev,
struct rte_eth_dev_info *dev_info);
+static int idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu);
static void idpf_adapter_rel(struct idpf_adapter *adapter);
static int
@@ -68,6 +69,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_release = idpf_dev_tx_queue_release,
.dev_infos_get = idpf_dev_info_get,
.link_update = idpf_dev_link_update,
+ .mtu_set = idpf_dev_mtu_set,
};
static int
@@ -110,6 +112,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 10/18] net/idpf: add support for basic Rx datapath
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (8 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 09/18] net/idpf: add support for MTU configuration Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
` (8 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 94f4671057..df2a760673 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -375,6 +375,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
goto err_vport;
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 11/18] net/idpf: add support for basic Tx datapath
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (9 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 12/18] net/idpf: support parsing packet type Junfeng Guo
` (7 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index df2a760673..6fb56e584d 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -88,6 +88,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -376,6 +378,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
if (idpf_vc_ena_dis_vport(vport, true) != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 12/18] net/idpf: support parsing packet type
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (10 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
` (6 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 6fb56e584d..630bdabcd4 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -709,6 +709,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 1ba59929a0..ac399d331a 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -980,3 +1194,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 13/18] net/idpf: add support for write back based on ITR expire
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (11 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 12/18] net/idpf: support parsing packet type Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 14/18] net/idpf: add support for RSS Junfeng Guo
` (5 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 111 +++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 630bdabcd4..8045073780 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -327,6 +327,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps here should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR. The if condition should
+ * be updated when the FW can return correct flag bits.
+ */
+ if (adapter->caps->other_caps != 0) {
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ }
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -364,6 +448,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
@@ -372,6 +460,23 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ goto err_mtu;
+ }
+ if (idpf_vc_alloc_vectors(vport, req_vecs_num) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ if (idpf_config_rx_queues_irqs(dev) != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
if (idpf_start_queues(dev) != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
goto err_mtu;
@@ -401,6 +506,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -412,6 +521,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -766,6 +880,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ac399d331a..dc629be741 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -1020,6 +1024,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 14/18] net/idpf: add support for RSS
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (12 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 15/18] net/idpf: add support for Rx offloading Junfeng Guo
` (4 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 116 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 26 ++++++++
drivers/net/idpf/idpf_vchnl.c | 97 +++++++++++++++++++++++++++
3 files changed, 239 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 8045073780..ea7d3bfd7e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -86,6 +86,7 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
@@ -198,6 +199,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -218,6 +221,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -275,10 +282,102 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
@@ -324,6 +423,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -1;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -521,6 +631,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index dc629be741..c354a5815f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -663,6 +666,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 15/18] net/idpf: add support for Rx offloading
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (13 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 14/18] net/idpf: add support for RSS Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 16/18] net/idpf: add support for Tx offloading Junfeng Guo
` (3 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
3 files changed, 134 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index c8f4a0d6ed..6eefac9529 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,9 +3,14 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
Queue start/stop = Y
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index ea7d3bfd7e..88f3db3267 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -89,6 +89,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 16/18] net/idpf: add support for Tx offloading
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (14 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 15/18] net/idpf: add support for Rx offloading Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
` (2 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li
Add Tx offloading support:
- support TSO
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 6eefac9529..86c182021f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -9,6 +9,7 @@
[Features]
Queue start/stop = Y
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 88f3db3267..868f28d003 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -95,7 +95,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..cc296d7ab1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
}
return i;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (15 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 16/18] net/idpf: add support for Tx offloading Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 18/18] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjun Wu
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 868f28d003..62b13b602e 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -97,7 +97,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cc296d7ab1..9e20f2b9d3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v14 18/18] net/idpf: add support for timestamp offload
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (16 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
@ 2022-10-27 7:47 ` Junfeng Guo
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: Junfeng Guo @ 2022-10-27 7:47 UTC (permalink / raw)
To: andrew.rybchenko, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Wenjing Qiao
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 86c182021f..5fbad9600c 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -12,6 +12,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 62b13b602e..84e7421d76 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -93,7 +95,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9e20f2b9d3..bafa007faf 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.34.1
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v11 15/18] net/idpf: add support for Rx offloading
2022-10-25 10:03 ` Andrew Rybchenko
@ 2022-10-28 1:48 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-28 1:48 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Tuesday, October 25, 2022 6:04 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v11 15/18] net/idpf: add support for Rx offloading
>
> On 10/24/22 16:12, Junfeng Guo wrote:
> > Add Rx offloading support:
> > - support CHKSUM and RSS offload for split queue model
> > - support CHKSUM offload for single queue model
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> > ---
> > doc/guides/nics/features/idpf.ini | 2 +
> > drivers/net/idpf/idpf_ethdev.c | 9 ++-
> > drivers/net/idpf/idpf_rxtx.c | 122 ++++++++++++++++++++++++++++++
> > 3 files changed, 132 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/nics/features/idpf.ini
> > b/doc/guides/nics/features/idpf.ini
> > index d4eb9b374c..c86d9378ea 100644
> > --- a/doc/guides/nics/features/idpf.ini
> > +++ b/doc/guides/nics/features/idpf.ini
> > @@ -9,5 +9,7 @@
> > [Features]
> > Queue start/stop = Y
> > MTU update = Y
> > +L3 checksum offload = P
> > +L4 checksum offload = P
>
> RSS hash missing
>
> > Packet type parsing = Y
> > Linux = Y
> > diff --git a/drivers/net/idpf/idpf_ethdev.c
> > b/drivers/net/idpf/idpf_ethdev.c index 739cf31d65..d8cc423a23 100644
> > --- a/drivers/net/idpf/idpf_ethdev.c
> > +++ b/drivers/net/idpf/idpf_ethdev.c
> > @@ -94,7 +94,14 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct
> rte_eth_dev_info *dev_info)
> > dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
> > dev_info->dev_capa =
> RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
> > RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
> > - dev_info->rx_offload_capa = 0;
> > +
> > + dev_info->rx_offload_capa =
> > + RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> > + RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
> > + RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
> > + RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
> > + RTE_ETH_RX_OFFLOAD_RSS_HASH;
>
> As I understand you know mode here and you should not report offload
> which is not supported in current mode (RSS vs single queue).
Yes, will remove RTE_ETH_RX_OFFLOAD_RSS_HASH here since RSS hash is not supported in single queue mode currently,
So we needn't to add 'RSS hash = Y' in idpf.ini, right?
>
> > +
> > dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
> >
> > dev_info->default_rxconf = (struct rte_eth_rxconf) {
>
> [snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v11 02/18] net/idpf: add support for device initialization
2022-10-25 8:57 ` Andrew Rybchenko
2022-10-26 8:28 ` Xing, Beilei
@ 2022-10-28 15:14 ` Andrew Rybchenko
2022-10-28 17:19 ` Xing, Beilei
1 sibling, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-28 15:14 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang
On 10/25/22 11:57, Andrew Rybchenko wrote:
> On 10/24/22 16:12, Junfeng Guo wrote:
>> Support device init and add the following dev ops:
>> - dev_configure
>> - dev_close
>> - dev_infos_get
>>
>> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
>> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
>> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
>> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
>> +struct idpf_adapter *
>> +idpf_find_adapter(struct rte_pci_device *pci_dev)
>
> It looks like the function requires corresponding lock to be
> held. If yes, it should be documented and code fixed. If no,
> it should be explaiend why.
I still don't understand it is a new patch. It is hardly safe to
return a pointer to an element list when you drop lock.
>> + /* valid only if rxq_model is split Q */
>> + uint16_t num_rx_bufq;
>> +
>> + uint16_t max_mtu;
>
> unused
Comments? It is still in place in a new version.
>> +int
>> +idpf_vc_get_caps(struct idpf_adapter *adapter)
>> +{
>> + struct virtchnl2_get_capabilities caps_msg;
>> + struct idpf_cmd_info args;
>> + int err;
>> +
>> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
>> + caps_msg.csum_caps =
>> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
>> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
>> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
>> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
>> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
>> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
>> +
>> + caps_msg.seg_caps =
>> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
>> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
>> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
>> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
>> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
>> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
>> + VIRTCHNL2_CAP_SEG_GENERIC;
>> +
>> + caps_msg.rss_caps =
>> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
>> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
>> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
>> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
>> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
>> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
>> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
>> + VIRTCHNL2_CAP_RSS_IPV4_AH |
>> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV6_AH |
>> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
>> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
>> +
>> + caps_msg.hsplit_caps =
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
>> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
>> +
>> + caps_msg.rsc_caps =
>> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
>> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
>> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
>> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
>> +
>> + caps_msg.other_caps =
>> + VIRTCHNL2_CAP_RDMA |
>> + VIRTCHNL2_CAP_SRIOV |
>> + VIRTCHNL2_CAP_MACFILTER |
>> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
>> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
>> + VIRTCHNL2_CAP_CRC |
>> + VIRTCHNL2_CAP_WB_ON_ITR |
>> + VIRTCHNL2_CAP_PROMISC |
>> + VIRTCHNL2_CAP_LINK_SPEED |
>> + VIRTCHNL2_CAP_VLAN;
>
> I'm wondering why all above capabilities are mentioned in the
> patch? What does the API do? Do it is request it? Negotiage?
Can I have answer on my question?
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v14 02/18] net/idpf: add support for device initialization
2022-10-27 7:47 ` [PATCH v14 02/18] net/idpf: add support for device initialization Junfeng Guo
@ 2022-10-28 15:35 ` Andrew Rybchenko
2022-10-28 17:22 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-28 15:35 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang, Wenjun Wu
On 10/27/22 10:47, Junfeng Guo wrote:
> Support device init and add the following dev ops:
> - dev_configure
> - dev_close
> - dev_infos_get
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> +static int idpf_dev_configure(struct rte_eth_dev *dev);
> +static int idpf_dev_close(struct rte_eth_dev *dev);
> +static int idpf_dev_info_get(struct rte_eth_dev *dev,
> + struct rte_eth_dev_info *dev_info);
> +static void idpf_adapter_rel(struct idpf_adapter *adapter);
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_close = idpf_dev_close,
> + .dev_infos_get = idpf_dev_info_get,
> +};
Typically it is better to avoid forward static declarations and
simply define the ops structure after callbacks.
> +
> +static int
> +idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> +
> + dev_info->max_rx_queues = adapter->caps->max_rx_q;
> + dev_info->max_tx_queues = adapter->caps->max_tx_q;
> + dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
> + dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
> +
> + dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
> + dev_info->min_mtu = RTE_ETHER_MIN_MTU;
> +
> + dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
I guess it make sense if and only if you support API
to add/remove unicast MAC addresses.
> +
> + return 0;
> +
[snip]
> +static int
> +idpf_init_vport(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> + struct idpf_adapter *adapter = vport->adapter;
> + uint16_t idx = adapter->cur_vport_idx;
> + struct virtchnl2_create_vport *vport_info =
> + (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
> + int i, type, ret;
> +
> + vport->vport_id = vport_info->vport_id;
> + vport->txq_model = vport_info->txq_model;
> + vport->rxq_model = vport_info->rxq_model;
> + vport->num_tx_q = vport_info->num_tx_q;
> + vport->num_tx_complq = vport_info->num_tx_complq;
> + vport->num_rx_q = vport_info->num_rx_q;
> + vport->num_rx_bufq = vport_info->num_rx_bufq;
> + vport->max_mtu = vport_info->max_mtu;
> + rte_memcpy(vport->default_mac_addr,
> + vport_info->default_mac_addr, ETH_ALEN);
> + vport->sw_idx = idx;
> +
> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> + type = vport_info->chunks.chunks[i].type;
> + switch (type) {
> + case VIRTCHNL2_QUEUE_TYPE_TX:
> + vport->chunks_info.tx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + break;
> + case VIRTCHNL2_QUEUE_TYPE_RX:
> + vport->chunks_info.rx_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + break;
> + case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
> + vport->chunks_info.tx_compl_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_compl_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_compl_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + break;
> + case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
> + vport->chunks_info.rx_buf_start_qid =
> + vport_info->chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_buf_qtail_start =
> + vport_info->chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_buf_qtail_spacing =
> + vport_info->chunks.chunks[i].qtail_reg_spacing;
> + break;
> + default:
> + PMD_INIT_LOG(ERR, "Unsupported queue type");
> + break;
> + }
> + }
> +
> + ret = idpf_parse_devarg_id(dev->data->name);
> + if (ret < 0) {
> + PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
> + return -1;
Negative errno must be returned since finally it is used as
rte_eth_dev_create() return value which is negative errno.
> + }
> + vport->devarg_id = ret;
> +
> + vport->dev_data = dev->data;
> +
> + adapter->vports[idx] = vport;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev)
> +{
> + struct rte_eth_conf *conf = &dev->data->dev_conf;
> +
> + if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
> + PMD_INIT_LOG(ERR, "Setting link speed is not supported");
> + return -1;
The return value is used as rte_eth_dev_configure() return value
which should be negative errno, not -1.
Please, double-check all other similar cases.
> + }
> +
> + if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
> + (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
Rigth now (just after the patch) you don't support RSS since you don't
handle corresponding configuratoin items. So, Nothing except RX_NONE
is supported. RX_RSS should be added when you really support it (later).
> + PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
> + conf->rxmode.mq_mode);
> + return -1;
> + }
[snip]
> +struct idpf_adapter *
> +idpf_find_adapter(struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter;
> +
> + rte_spinlock_lock(&idpf_adapter_lock);
> + TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
> + if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
> + rte_spinlock_unlock(&idpf_adapter_lock);
> + return adapter;
Pointer to an element of the list protected by spin lock is returned
here.
> + }
> + }
> + rte_spinlock_unlock(&idpf_adapter_lock);
> +
> + return NULL;
> +}
[snip]
> +static int
> +idpf_pci_remove(struct rte_pci_device *pci_dev)
> +{
> + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
Question about locking still stands. I'm not sure that I understand
the purpose of locking here. Or why can it be omitted.
Anyway, returing pointer to the list element when the list is
protected by lock looks suspicous.
> + uint16_t port_id;
> +
> + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
> + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
> + rte_eth_dev_close(port_id);
> + }
> +
> + rte_spinlock_lock(&idpf_adapter_lock);
> + TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
> + rte_spinlock_unlock(&idpf_adapter_lock);
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> +
> + return 0;
> +}
[snip]
> +int
> +idpf_vc_get_caps(struct idpf_adapter *adapter)
> +{
> + struct virtchnl2_get_capabilities caps_msg;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> + caps_msg.csum_caps =
> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> +
> + caps_msg.seg_caps =
> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> + VIRTCHNL2_CAP_SEG_GENERIC;
> +
> + caps_msg.rss_caps =
> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> +
> + caps_msg.other_caps =
> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> + VIRTCHNL2_CAP_CRC |
> + VIRTCHNL2_CAP_WB_ON_ITR |
> + VIRTCHNL2_CAP_PROMISC |
> + VIRTCHNL2_CAP_LINK_SPEED |
> + VIRTCHNL2_CAP_VLAN;
> +
My question aske in v11 still stands since I have no an answer yet.
Basically it looks like corresponding caps should be added when
corresponding offload support is added later. If no, I'd like to
understand why.
> + args.ops = VIRTCHNL2_OP_GET_CAPS;
> + args.in_args = (uint8_t *)&caps_msg;
> + args.in_args_size = sizeof(caps_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err != 0) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
> + return err;
> + }
> +
> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> +
> + return 0;
> +}
[snip]
> +int
> +idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
> + VIRTCHNL2_OP_DISABLE_VPORT;
> + args.in_args = (u8 *)&vc_vport;
uint8_t should be used as you do above in idpf_vc_destroy_vport() and
many other cases.
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v14 05/18] net/idpf: add support for device start and stop
2022-10-27 7:47 ` [PATCH v14 05/18] net/idpf: add support for device start and stop Junfeng Guo
@ 2022-10-28 15:45 ` Andrew Rybchenko
0 siblings, 0 replies; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-28 15:45 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/27/22 10:47, Junfeng Guo wrote:
> Add dev ops dev_start, dev_stop and link_update.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> @@ -284,6 +305,40 @@ idpf_dev_configure(struct rte_eth_dev *dev)
> return 0;
> }
>
> +static int
> +idpf_dev_start(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport = dev->data->dev_private;
> +
> + if (dev->data->mtu > vport->max_mtu) {
> + PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
> + return -1;
Negative errno must be returned.
> + }
> +
> + vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
> +
> + /* TODO: start queues */
> +
> + if (idpf_vc_ena_dis_vport(vport, true) != 0) {
> + PMD_DRV_LOG(ERR, "Failed to enable vport");
> + return -1;
same here
> + }
> +
> + return 0;
> +}
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v14 06/18] net/idpf: add support for queue start
2022-10-27 7:47 ` [PATCH v14 06/18] net/idpf: add support for queue start Junfeng Guo
@ 2022-10-28 15:50 ` Andrew Rybchenko
2022-10-28 17:34 ` Xing, Beilei
0 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-28 15:50 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing; +Cc: dev, Xiaoyun Li
On 10/27/22 10:47, Junfeng Guo wrote:
> Add support for these device ops:
> - rx_queue_start
> - tx_queue_start
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[snip]
> +#define IDPF_RX_BUF_STRIDE 64
> +int
> +idpf_vc_config_rxqs(struct idpf_vport *vport)
> +{
> + struct idpf_adapter *adapter = vport->adapter;
> + struct idpf_rx_queue **rxq =
> + (struct idpf_rx_queue **)vport->dev_data->rx_queues;
> + struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
> + struct virtchnl2_rxq_info *rxq_info;
> + struct idpf_cmd_info args;
> + uint16_t total_qs, num_qs;
> + int size, i, j;
> + int err = 0;
> + int k = 0;
> +
> + total_qs = vport->num_rx_q + vport->num_rx_bufq;
> + while (total_qs) {
> + if (total_qs > adapter->max_rxq_per_msg) {
> + num_qs = adapter->max_rxq_per_msg;
> + total_qs -= adapter->max_rxq_per_msg;
> + } else {
> + num_qs = total_qs;
> + total_qs = 0;
> + }
> +
> + size = sizeof(*vc_rxqs) + (num_qs - 1) *
> + sizeof(struct virtchnl2_rxq_info);
> + vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
> + if (vc_rxqs == NULL) {
> + PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
> + err = -ENOMEM;
> + break;
> + }
> + vc_rxqs->vport_id = vport->vport_id;
> + vc_rxqs->num_qinfo = num_qs;
> + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
> + for (i = 0; i < num_qs; i++, k++) {
> + rxq_info = &vc_rxqs->qinfo[i];
> + rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
> + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
> + rxq_info->queue_id = rxq[k]->queue_id;
> + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
> + rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
> + rxq_info->max_pkt_size = vport->max_pkt_len;
> +
> + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
> + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
> +
> + rxq_info->ring_len = rxq[k]->nb_rx_desc;
> + }
> + } else {
> + for (i = 0; i < num_qs / 3; i++, k++) {
> + /* Rx queue */
> + rxq_info = &vc_rxqs->qinfo[i * 3];
> + rxq_info->dma_ring_addr =
> + rxq[k]->rx_ring_phys_addr;
> + rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
> + rxq_info->queue_id = rxq[k]->queue_id;
> + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
> + rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
> + rxq_info->max_pkt_size = vport->max_pkt_len;
> +
> + rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
> + rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
> +
> + rxq_info->ring_len = rxq[k]->nb_rx_desc;
> + rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
> + rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
> + rxq_info->rx_buffer_low_watermark = 64;
> +
> + /* Buffer queue */
> + for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
> + struct idpf_rx_queue *bufq = j == 1 ?
> + rxq[k]->bufq1 : rxq[k]->bufq2;
> + rxq_info = &vc_rxqs->qinfo[i * 3 + j];
> + rxq_info->dma_ring_addr =
> + bufq->rx_ring_phys_addr;
> + rxq_info->type =
> + VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
> + rxq_info->queue_id = bufq->queue_id;
> + rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
> + rxq_info->data_buffer_size = bufq->rx_buf_len;
> + rxq_info->desc_ids =
> + VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
> + rxq_info->ring_len = bufq->nb_rx_desc;
> +
> + rxq_info->buffer_notif_stride =
> + IDPF_RX_BUF_STRIDE;
> + rxq_info->rx_buffer_low_watermark = 64;
> + }
> + }
> + }
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
> + args.in_args = (uint8_t *)vc_rxqs;
> + args.in_args_size = size;
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
The function must return negative errno since its return
value is used as a return value of functions returning
negative errno.
> + rte_free(vc_rxqs);
> + if (err != 0) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
> + break;
> + }
> + }
> +
> + return err;
> +}
[snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v11 02/18] net/idpf: add support for device initialization
2022-10-28 15:14 ` Andrew Rybchenko
@ 2022-10-28 17:19 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-28 17:19 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun, Wang, Xiao W
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, October 28, 2022 11:14 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v11 02/18] net/idpf: add support for device initialization
>
> On 10/25/22 11:57, Andrew Rybchenko wrote:
> > On 10/24/22 16:12, Junfeng Guo wrote:
> >> Support device init and add the following dev ops:
> >> - dev_configure
> >> - dev_close
> >> - dev_infos_get
> >>
> >> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> >> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> >> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> >> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> >> +struct idpf_adapter *
> >> +idpf_find_adapter(struct rte_pci_device *pci_dev)
> >
> > It looks like the function requires corresponding lock to be held. If
> > yes, it should be documented and code fixed. If no, it should be
> > explaiend why.
>
> I still don't understand it is a new patch. It is hardly safe to return a pointer
> to an element list when you drop lock.
Sorry I misunderstood your last comment, I thought you meant lock for adapter_list.
I don't think we need a lock for adapter, one adapter here doesn't map one ethdev,
but one PCI device, we can create some vports for one adapter, and one vport maps
one ethdev.
>
> >> + /* valid only if rxq_model is split Q */
> >> + uint16_t num_rx_bufq;
> >> +
> >> + uint16_t max_mtu;
> >
> > unused
>
> Comments? It is still in place in a new version.
All the above info is returned by backend when creating a vport, so save it after creating vport.
>
> >> +int
> >> +idpf_vc_get_caps(struct idpf_adapter *adapter) {
> >> + struct virtchnl2_get_capabilities caps_msg;
> >> + struct idpf_cmd_info args;
> >> + int err;
> >> +
> >> + memset(&caps_msg, 0, sizeof(struct
> >> +virtchnl2_get_capabilities));
> >> + caps_msg.csum_caps =
> >> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> >> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> >> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> >> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> >> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> >> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> >> +
> >> + caps_msg.seg_caps =
> >> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> >> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> >> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> >> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> >> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> >> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> >> + VIRTCHNL2_CAP_SEG_GENERIC;
> >> +
> >> + caps_msg.rss_caps =
> >> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> >> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> >> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> >> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> >> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> >> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> >> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> >> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> >> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> >> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> >> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> >> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> >> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> >> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> >> +
> >> + caps_msg.hsplit_caps =
> >> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
> >> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
> >> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
> >> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
> >> +
> >> + caps_msg.rsc_caps =
> >> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
> >> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
> >> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
> >> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
> >> +
> >> + caps_msg.other_caps =
> >> + VIRTCHNL2_CAP_RDMA |
> >> + VIRTCHNL2_CAP_SRIOV |
> >> + VIRTCHNL2_CAP_MACFILTER |
> >> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
> >> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> >> + VIRTCHNL2_CAP_CRC |
> >> + VIRTCHNL2_CAP_WB_ON_ITR |
> >> + VIRTCHNL2_CAP_PROMISC |
> >> + VIRTCHNL2_CAP_LINK_SPEED |
> >> + VIRTCHNL2_CAP_VLAN;
> >
> > I'm wondering why all above capabilities are mentioned in the patch?
> > What does the API do? Do it is request it? Negotiage?
>
> Can I have answer on my question?
I removed some caps in v14, and I think it makes sense adding one cap when enable the offload.
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v14 02/18] net/idpf: add support for device initialization
2022-10-28 15:35 ` Andrew Rybchenko
@ 2022-10-28 17:22 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-28 17:22 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun, Wang, Xiao W, Wu, Wenjun1
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, October 28, 2022 11:35 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; Wu, Wenjun1 <wenjun1.wu@intel.com>
> Subject: Re: [PATCH v14 02/18] net/idpf: add support for device initialization
>
> On 10/27/22 10:47, Junfeng Guo wrote:
> > Support device init and add the following dev ops:
> > - dev_configure
> > - dev_close
> > - dev_infos_get
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > +static int idpf_dev_configure(struct rte_eth_dev *dev); static int
> > +idpf_dev_close(struct rte_eth_dev *dev); static int
> > +idpf_dev_info_get(struct rte_eth_dev *dev,
> > + struct rte_eth_dev_info *dev_info); static void
> > +idpf_adapter_rel(struct idpf_adapter *adapter);
> > +
> > +static const struct eth_dev_ops idpf_eth_dev_ops = {
> > + .dev_configure = idpf_dev_configure,
> > + .dev_close = idpf_dev_close,
> > + .dev_infos_get = idpf_dev_info_get,
> > +};
>
> Typically it is better to avoid forward static declarations and simply define
> the ops structure after callbacks.
OK, will fix it in v15.
>
> > +
> > +static int
> > +idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info
> > +*dev_info) {
> > + struct idpf_vport *vport = dev->data->dev_private;
> > + struct idpf_adapter *adapter = vport->adapter;
> > +
> > + dev_info->max_rx_queues = adapter->caps->max_rx_q;
> > + dev_info->max_tx_queues = adapter->caps->max_tx_q;
> > + dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
> > + dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
> > +
> > + dev_info->max_mtu = dev_info->max_rx_pktlen -
> IDPF_ETH_OVERHEAD;
> > + dev_info->min_mtu = RTE_ETHER_MIN_MTU;
> > +
> > + dev_info->max_mac_addrs = IDPF_NUM_MACADDR_MAX;
>
> I guess it make sense if and only if you support API to add/remove unicast
> MAC addresses.
Yes, will remove this info.
>
> > +
> > + return 0;
> > +
>
> [snip]
>
> > +static int
> > +idpf_init_vport(struct rte_eth_dev *dev) {
> > + struct idpf_vport *vport = dev->data->dev_private;
> > + struct idpf_adapter *adapter = vport->adapter;
> > + uint16_t idx = adapter->cur_vport_idx;
> > + struct virtchnl2_create_vport *vport_info =
> > + (struct virtchnl2_create_vport *)adapter-
> >vport_recv_info[idx];
> > + int i, type, ret;
> > +
> > + vport->vport_id = vport_info->vport_id;
> > + vport->txq_model = vport_info->txq_model;
> > + vport->rxq_model = vport_info->rxq_model;
> > + vport->num_tx_q = vport_info->num_tx_q;
> > + vport->num_tx_complq = vport_info->num_tx_complq;
> > + vport->num_rx_q = vport_info->num_rx_q;
> > + vport->num_rx_bufq = vport_info->num_rx_bufq;
> > + vport->max_mtu = vport_info->max_mtu;
> > + rte_memcpy(vport->default_mac_addr,
> > + vport_info->default_mac_addr, ETH_ALEN);
> > + vport->sw_idx = idx;
> > +
> > + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> > + type = vport_info->chunks.chunks[i].type;
> > + switch (type) {
> > + case VIRTCHNL2_QUEUE_TYPE_TX:
> > + vport->chunks_info.tx_start_qid =
> > + vport_info->chunks.chunks[i].start_queue_id;
> > + vport->chunks_info.tx_qtail_start =
> > + vport_info->chunks.chunks[i].qtail_reg_start;
> > + vport->chunks_info.tx_qtail_spacing =
> > + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> > + break;
> > + case VIRTCHNL2_QUEUE_TYPE_RX:
> > + vport->chunks_info.rx_start_qid =
> > + vport_info->chunks.chunks[i].start_queue_id;
> > + vport->chunks_info.rx_qtail_start =
> > + vport_info->chunks.chunks[i].qtail_reg_start;
> > + vport->chunks_info.rx_qtail_spacing =
> > + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> > + break;
> > + case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
> > + vport->chunks_info.tx_compl_start_qid =
> > + vport_info->chunks.chunks[i].start_queue_id;
> > + vport->chunks_info.tx_compl_qtail_start =
> > + vport_info->chunks.chunks[i].qtail_reg_start;
> > + vport->chunks_info.tx_compl_qtail_spacing =
> > + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> > + break;
> > + case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
> > + vport->chunks_info.rx_buf_start_qid =
> > + vport_info->chunks.chunks[i].start_queue_id;
> > + vport->chunks_info.rx_buf_qtail_start =
> > + vport_info->chunks.chunks[i].qtail_reg_start;
> > + vport->chunks_info.rx_buf_qtail_spacing =
> > + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> > + break;
> > + default:
> > + PMD_INIT_LOG(ERR, "Unsupported queue type");
> > + break;
> > + }
> > + }
> > +
> > + ret = idpf_parse_devarg_id(dev->data->name);
> > + if (ret < 0) {
> > + PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
> > + return -1;
>
> Negative errno must be returned since finally it is used as
> rte_eth_dev_create() return value which is negative errno.
Make sense, will refine the return value, and check all similar cases.
>
> > + }
> > + vport->devarg_id = ret;
> > +
> > + vport->dev_data = dev->data;
> > +
> > + adapter->vports[idx] = vport;
> > +
> > + return 0;
> > +}
> > +
> > +static int
> > +idpf_dev_configure(struct rte_eth_dev *dev) {
> > + struct rte_eth_conf *conf = &dev->data->dev_conf;
> > +
> > + if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
> > + PMD_INIT_LOG(ERR, "Setting link speed is not supported");
> > + return -1;
>
> The return value is used as rte_eth_dev_configure() return value which
> should be negative errno, not -1.
>
> Please, double-check all other similar cases.
>
> > + }
> > +
> > + if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode !=
> RTE_ETH_MQ_RX_NONE) ||
> > + (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode !=
> > +RTE_ETH_MQ_RX_RSS)) {
>
> Rigth now (just after the patch) you don't support RSS since you don't
> handle corresponding configuratoin items. So, Nothing except RX_NONE is
> supported. RX_RSS should be added when you really support it (later).
>
> > + PMD_INIT_LOG(ERR, "Multi-queue packet distribution
> mode %d is not supported",
> > + conf->rxmode.mq_mode);
> > + return -1;
> > + }
>
> [snip]
>
> > +struct idpf_adapter *
> > +idpf_find_adapter(struct rte_pci_device *pci_dev) {
> > + struct idpf_adapter *adapter;
> > +
> > + rte_spinlock_lock(&idpf_adapter_lock);
> > + TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
> > + if (strncmp(adapter->name, pci_dev->device.name,
> PCI_PRI_STR_SIZE) == 0) {
> > + rte_spinlock_unlock(&idpf_adapter_lock);
> > + return adapter;
>
> Pointer to an element of the list protected by spin lock is returned here.
Yes, will fix it in v15.
>
> > + }
> > + }
> > + rte_spinlock_unlock(&idpf_adapter_lock);
> > +
> > + return NULL;
> > +}
>
> [snip]
>
> > +static int
> > +idpf_pci_remove(struct rte_pci_device *pci_dev) {
> > + struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
>
> Question about locking still stands. I'm not sure that I understand the
> purpose of locking here. Or why can it be omitted.
> Anyway, returing pointer to the list element when the list is protected by
> lock looks suspicous.
>
> > + uint16_t port_id;
> > +
> > + /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF
> through rte_device */
> > + RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
> > + rte_eth_dev_close(port_id);
> > + }
> > +
> > + rte_spinlock_lock(&idpf_adapter_lock);
> > + TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
> > + rte_spinlock_unlock(&idpf_adapter_lock);
> > + idpf_adapter_rel(adapter);
> > + rte_free(adapter);
> > +
> > + return 0;
> > +}
>
> [snip]
>
> > +int
> > +idpf_vc_get_caps(struct idpf_adapter *adapter) {
> > + struct virtchnl2_get_capabilities caps_msg;
> > + struct idpf_cmd_info args;
> > + int err;
> > +
> > + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> > + caps_msg.csum_caps =
> > + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> > + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> > + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> > + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> > + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> > + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> > +
> > + caps_msg.seg_caps =
> > + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> > + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> > + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> > + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> > + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> > + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> > + VIRTCHNL2_CAP_SEG_GENERIC;
> > +
> > + caps_msg.rss_caps =
> > + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> > + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> > + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> > + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> > + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> > + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> > + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> > + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> > + VIRTCHNL2_CAP_RSS_IPV4_AH |
> > + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> > + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> > + VIRTCHNL2_CAP_RSS_IPV6_AH |
> > + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> > + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> > +
> > + caps_msg.other_caps =
> > + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> > + VIRTCHNL2_CAP_CRC |
> > + VIRTCHNL2_CAP_WB_ON_ITR |
> > + VIRTCHNL2_CAP_PROMISC |
> > + VIRTCHNL2_CAP_LINK_SPEED |
> > + VIRTCHNL2_CAP_VLAN;
> > +
>
> My question aske in v11 still stands since I have no an answer yet.
> Basically it looks like corresponding caps should be added when
> corresponding offload support is added later. If no, I'd like to understand
> why.
>
> > + args.ops = VIRTCHNL2_OP_GET_CAPS;
> > + args.in_args = (uint8_t *)&caps_msg;
> > + args.in_args_size = sizeof(caps_msg);
> > + args.out_buffer = adapter->mbx_resp;
> > + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> > +
> > + err = idpf_execute_vc_cmd(adapter, &args);
> > + if (err != 0) {
> > + PMD_DRV_LOG(ERR,
> > + "Failed to execute command of
> VIRTCHNL2_OP_GET_CAPS");
> > + return err;
> > + }
> > +
> > + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> > +
> > + return 0;
> > +}
>
> [snip]
>
> > +int
> > +idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable) {
> > + struct idpf_adapter *adapter = vport->adapter;
> > + struct virtchnl2_vport vc_vport;
> > + struct idpf_cmd_info args;
> > + int err;
> > +
> > + vc_vport.vport_id = vport->vport_id;
> > + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
> > + VIRTCHNL2_OP_DISABLE_VPORT;
> > + args.in_args = (u8 *)&vc_vport;
>
> uint8_t should be used as you do above in idpf_vc_destroy_vport() and
> many other cases.
>
> [snip]
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v14 06/18] net/idpf: add support for queue start
2022-10-28 15:50 ` Andrew Rybchenko
@ 2022-10-28 17:34 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-28 17:34 UTC (permalink / raw)
To: Andrew Rybchenko, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, October 28, 2022 11:51 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>
> Subject: Re: [PATCH v14 06/18] net/idpf: add support for queue start
>
> On 10/27/22 10:47, Junfeng Guo wrote:
> > Add support for these device ops:
> > - rx_queue_start
> > - tx_queue_start
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
>
> [snip]
>
> > +#define IDPF_RX_BUF_STRIDE 64
> > +int
> > +idpf_vc_config_rxqs(struct idpf_vport *vport) {
> > + struct idpf_adapter *adapter = vport->adapter;
> > + struct idpf_rx_queue **rxq =
> > + (struct idpf_rx_queue **)vport->dev_data->rx_queues;
> > + struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
> > + struct virtchnl2_rxq_info *rxq_info;
> > + struct idpf_cmd_info args;
> > + uint16_t total_qs, num_qs;
> > + int size, i, j;
> > + int err = 0;
> > + int k = 0;
> > +
> > + total_qs = vport->num_rx_q + vport->num_rx_bufq;
> > + while (total_qs) {
> > + if (total_qs > adapter->max_rxq_per_msg) {
> > + num_qs = adapter->max_rxq_per_msg;
> > + total_qs -= adapter->max_rxq_per_msg;
> > + } else {
> > + num_qs = total_qs;
> > + total_qs = 0;
> > + }
> > +
> > + size = sizeof(*vc_rxqs) + (num_qs - 1) *
> > + sizeof(struct virtchnl2_rxq_info);
> > + vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
> > + if (vc_rxqs == NULL) {
> > + PMD_DRV_LOG(ERR, "Failed to allocate
> virtchnl2_config_rx_queues");
> > + err = -ENOMEM;
> > + break;
> > + }
> > + vc_rxqs->vport_id = vport->vport_id;
> > + vc_rxqs->num_qinfo = num_qs;
> > + if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
> {
> > + for (i = 0; i < num_qs; i++, k++) {
> > + rxq_info = &vc_rxqs->qinfo[i];
> > + rxq_info->dma_ring_addr = rxq[k]-
> >rx_ring_phys_addr;
> > + rxq_info->type =
> VIRTCHNL2_QUEUE_TYPE_RX;
> > + rxq_info->queue_id = rxq[k]->queue_id;
> > + rxq_info->model =
> VIRTCHNL2_QUEUE_MODEL_SINGLE;
> > + rxq_info->data_buffer_size = rxq[k]-
> >rx_buf_len;
> > + rxq_info->max_pkt_size = vport-
> >max_pkt_len;
> > +
> > + rxq_info->desc_ids =
> VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
> > + rxq_info->qflags |=
> VIRTCHNL2_RX_DESC_SIZE_32BYTE;
> > +
> > + rxq_info->ring_len = rxq[k]->nb_rx_desc;
> > + }
> > + } else {
> > + for (i = 0; i < num_qs / 3; i++, k++) {
> > + /* Rx queue */
> > + rxq_info = &vc_rxqs->qinfo[i * 3];
> > + rxq_info->dma_ring_addr =
> > + rxq[k]->rx_ring_phys_addr;
> > + rxq_info->type =
> VIRTCHNL2_QUEUE_TYPE_RX;
> > + rxq_info->queue_id = rxq[k]->queue_id;
> > + rxq_info->model =
> VIRTCHNL2_QUEUE_MODEL_SPLIT;
> > + rxq_info->data_buffer_size = rxq[k]-
> >rx_buf_len;
> > + rxq_info->max_pkt_size = vport-
> >max_pkt_len;
> > +
> > + rxq_info->desc_ids =
> VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
> > + rxq_info->qflags |=
> VIRTCHNL2_RX_DESC_SIZE_32BYTE;
> > +
> > + rxq_info->ring_len = rxq[k]->nb_rx_desc;
> > + rxq_info->rx_bufq1_id = rxq[k]->bufq1-
> >queue_id;
> > + rxq_info->rx_bufq2_id = rxq[k]->bufq2-
> >queue_id;
> > + rxq_info->rx_buffer_low_watermark = 64;
> > +
> > + /* Buffer queue */
> > + for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
> > + struct idpf_rx_queue *bufq = j == 1 ?
> > + rxq[k]->bufq1 : rxq[k]->bufq2;
> > + rxq_info = &vc_rxqs->qinfo[i * 3 + j];
> > + rxq_info->dma_ring_addr =
> > + bufq->rx_ring_phys_addr;
> > + rxq_info->type =
> > +
> VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
> > + rxq_info->queue_id = bufq-
> >queue_id;
> > + rxq_info->model =
> VIRTCHNL2_QUEUE_MODEL_SPLIT;
> > + rxq_info->data_buffer_size = bufq-
> >rx_buf_len;
> > + rxq_info->desc_ids =
> > +
> VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
> > + rxq_info->ring_len = bufq-
> >nb_rx_desc;
> > +
> > + rxq_info->buffer_notif_stride =
> > + IDPF_RX_BUF_STRIDE;
> > + rxq_info->rx_buffer_low_watermark
> = 64;
> > + }
> > + }
> > + }
> > + memset(&args, 0, sizeof(args));
> > + args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
> > + args.in_args = (uint8_t *)vc_rxqs;
> > + args.in_args_size = size;
> > + args.out_buffer = adapter->mbx_resp;
> > + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> > +
> > + err = idpf_execute_vc_cmd(adapter, &args);
>
> The function must return negative errno since its return value is used as a
> return value of functions returning negative errno.
Thanks for all your comments, please expect v15 tomorrow..
>
> > + rte_free(vc_rxqs);
> > + if (err != 0) {
> > + PMD_DRV_LOG(ERR, "Failed to execute command of
> VIRTCHNL2_OP_CONFIG_RX_QUEUES");
> > + break;
> > + }
> > + }
> > +
> > + return err;
> > +}
>
> [snip]
>
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 00/18] add support for idpf PMD in DPDK
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
` (17 preceding siblings ...)
2022-10-27 7:47 ` [PATCH v14 18/18] net/idpf: add support for timestamp offload Junfeng Guo
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 01/18] common/idpf: introduce common library beilei.xing
` (19 more replies)
18 siblings, 20 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev
From: Beilei Xing <beilei.xing@intel.com>
This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style according to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
v14:
- fix wrong position for rte_validate_tx_offload
v15:
- refine the return value for ethdev ops.
- removce forward static declarations.
- refine get caps.
- fix lock/unlock handling.
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 17 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1293 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1416 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15255 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 01/18] common/idpf: introduce common library
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 02/18] net/idpf: add support for device initialization beilei.xing
` (18 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiao Wang
From: Junfeng Guo <junfeng.guo@intel.com>
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 234 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8502 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..9efb0ea37a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = IDPF_SUCCESS;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = IDPF_ERR_DEVICE_NOT_SUPPORTED;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = IDPF_SUCCESS;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return IDPF_ERR_NO_MEMORY;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status != IDPF_SUCCESS) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ IDPF_SUCCESS, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..da3692df7d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = IDPF_SUCCESS;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return IDPF_ERR_CFG;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return IDPF_ERR_NO_MEMORY;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = IDPF_ERR_PARAM;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = IDPF_ERR_NO_MEMORY;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = IDPF_SUCCESS;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = IDPF_ERR_CTLQ_FULL;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = IDPF_SUCCESS;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*clean_count == 0)
+ return IDPF_SUCCESS;
+ if (*clean_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = IDPF_SUCCESS;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = IDPF_SUCCESS;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return IDPF_ERR_CTLQ_EMPTY;
+
+ if (*num_q_msg == 0)
+ return IDPF_SUCCESS;
+ else if (*num_q_msg > cq->ring_size)
+ return IDPF_ERR_PARAM;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = IDPF_ERR_CTLQ_ERROR;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = IDPF_ERR_CTLQ_NO_WORK;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..3bb8f72aad
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,234 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+/* Error Codes */
+/* Linux kernel driver can't directly use these. Instead, they are mapped to
+ * linux compatible error codes which get translated in the build script.
+ */
+#define IDPF_SUCCESS 0
+#define IDPF_ERR_PARAM -53 /* -EBADR */
+#define IDPF_ERR_NOT_IMPL -95 /* -EOPNOTSUPP */
+#define IDPF_ERR_NOT_READY -16 /* -EBUSY */
+#define IDPF_ERR_BAD_PTR -14 /* -EFAULT */
+#define IDPF_ERR_INVAL_SIZE -90 /* -EMSGSIZE */
+#define IDPF_ERR_DEVICE_NOT_SUPPORTED -19 /* -ENODEV */
+#define IDPF_ERR_FW_API_VER -13 /* -EACCESS */
+#define IDPF_ERR_NO_MEMORY -12 /* -ENOMEM */
+#define IDPF_ERR_CFG -22 /* -EINVAL */
+#define IDPF_ERR_OUT_OF_RANGE -34 /* -ERANGE */
+#define IDPF_ERR_ALREADY_EXISTS -17 /* -EEXIST */
+#define IDPF_ERR_DOES_NOT_EXIST -6 /* -ENXIO */
+#define IDPF_ERR_IN_USE -114 /* -EALREADY */
+#define IDPF_ERR_MAX_LIMIT -109 /* -ETOOMANYREFS */
+#define IDPF_ERR_RESET_ONGOING -104 /* -ECONNRESET */
+
+/* CRQ/CSQ specific error codes */
+#define IDPF_ERR_CTLQ_ERROR -74 /* -EBADMSG */
+#define IDPF_ERR_CTLQ_TIMEOUT -110 /* -ETIMEDOUT */
+#define IDPF_ERR_CTLQ_FULL -28 /* -ENOSPC */
+#define IDPF_ERR_CTLQ_NO_WORK -42 /* -ENOMSG */
+#define IDPF_ERR_CTLQ_EMPTY -105 /* -ENOBUFS */
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..00dfedb6a4
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return IDPF_ERR_NO_MEMORY;
+
+ return IDPF_SUCCESS;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return IDPF_SUCCESS;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return IDPF_ERR_NO_MEMORY;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return IDPF_SUCCESS;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return IDPF_ERR_NO_MEMORY;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return IDPF_ERR_CFG;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return IDPF_SUCCESS;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 02/18] net/idpf: add support for device initialization
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-29 3:27 ` [PATCH v15 01/18] common/idpf: introduce common library beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 03/18] net/idpf: add Tx queue setup beilei.xing
` (17 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 891 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 416 ++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1662 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index bdf233c9f8..cc66db25e8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -770,6 +770,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 4d40ea29a3..12841ce407 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -34,6 +34,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index a6f180ed99..7b66733aea 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -165,6 +165,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..035f563275
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,891 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -EINVAL;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -ENOMEM;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -EINVAL;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -EINVAL;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -EINVAL;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ ret = -ENOMEM;
+ goto err_mbx;
+ }
+
+ ret = idpf_vc_check_api_version(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ ret = -ENOMEM;
+ goto err_api;
+ }
+
+ ret = idpf_vc_get_caps(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ ret = -ENOMEM;
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ ret = -ENOMEM;
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ ret = -ENOMEM;
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return ret;
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ int found = 0;
+
+ if (pci_dev == NULL)
+ return NULL;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ if (found == 0)
+ return NULL;
+
+ return adapter;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -ENOMEM;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..3c685c2d0a
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != IDPF_ERR_CTLQ_NO_WORK)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -EINVAL;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -ENOMEM;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 589399400a..6470bf3636 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -29,6 +29,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 03/18] net/idpf: add Tx queue setup
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-29 3:27 ` [PATCH v15 01/18] common/idpf: introduce common library beilei.xing
2022-10-29 3:27 ` [PATCH v15 02/18] net/idpf: add support for device initialization beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 04/18] net/idpf: add Rx " beilei.xing
` (16 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/meson.build | 1 +
4 files changed, 448 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 035f563275..54f20d30ca 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -42,6 +43,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -631,6 +643,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 04/18] net/idpf: add Rx queue setup
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (2 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 03/18] net/idpf: add Tx queue setup beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 05/18] net/idpf: add support for device start and stop beilei.xing
` (15 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 54f20d30ca..fb5cd1b111 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -48,12 +48,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -643,6 +653,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 05/18] net/idpf: add support for device start and stop
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (3 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 04/18] net/idpf: add Rx " beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 06/18] net/idpf: add support for queue start beilei.xing
` (14 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index fb5cd1b111..621bf9aad5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,6 +29,22 @@ static const char * const idpf_valid_args[] = {
NULL
};
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static int
idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
{
@@ -267,6 +283,42 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ int ret;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -EINVAL;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ ret = idpf_vc_ena_dis_vport(vport, true);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
@@ -656,6 +708,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .link_update = idpf_dev_link_update,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 06/18] net/idpf: add support for queue start
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (4 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 05/18] net/idpf: add support for device start and stop beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 07/18] net/idpf: add support for queue stop beilei.xing
` (13 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 42 +++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 720 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 621bf9aad5..0400ed611f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -283,6 +283,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -296,11 +329,16 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ ret = idpf_start_queues(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return ret;
+ }
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
+ /* TODO: stop queues */
return ret;
}
@@ -711,6 +749,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.link_update = idpf_dev_link_update,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 3c685c2d0a..ff971c4cb5 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -390,6 +395,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 07/18] net/idpf: add support for queue stop
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (5 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 06/18] net/idpf: add support for queue start beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 08/18] net/idpf: add queue release beilei.xing
` (12 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 17 ++--
drivers/net/idpf/idpf_rxtx.c | 148 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
4 files changed, 242 insertions(+), 5 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0400ed611f..9f1e1e6a18 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -324,7 +324,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
@@ -332,17 +333,21 @@ idpf_dev_start(struct rte_eth_dev *dev)
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return ret;
+ goto err_mtu;
}
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- /* TODO: stop queues */
- return ret;
+ goto err_vport;
}
return 0;
+
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return ret;
}
static int
@@ -352,7 +357,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
@@ -751,6 +756,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.link_update = idpf_dev_link_update,
.rx_queue_start = idpf_rx_queue_start,
.tx_queue_start = idpf_tx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_stop = idpf_tx_queue_stop,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ff971c4cb5..0f95d4ee85 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -837,6 +837,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 08/18] net/idpf: add queue release
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (6 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 07/18] net/idpf: add support for queue stop beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 09/18] net/idpf: add support for MTU configuration beilei.xing
` (11 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 9f1e1e6a18..1485f40e71 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -758,6 +758,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 09/18] net/idpf: add support for MTU configuration
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (7 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 08/18] net/idpf: add queue release beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 10/18] net/idpf: add support for basic Rx datapath beilei.xing
` (10 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 13 +++++++++++++
2 files changed, 14 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..d722c49fde 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1485f40e71..856f3d7266 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -760,6 +772,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .mtu_set = idpf_dev_mtu_set,
};
static uint16_t
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 10/18] net/idpf: add support for basic Rx datapath
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (8 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 09/18] net/idpf: add support for MTU configuration beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 11/18] net/idpf: add support for basic Tx datapath beilei.xing
` (9 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 856f3d7266..2f1f9eeee5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -348,6 +348,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 11/18] net/idpf: add support for basic Tx datapath
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (9 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 10/18] net/idpf: add support for basic Rx datapath beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 12/18] net/idpf: support parsing packet type beilei.xing
` (8 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 2f1f9eeee5..f9f6fe1162 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -349,6 +351,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 12/18] net/idpf: support parsing packet type
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (10 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 11/18] net/idpf: add support for basic Tx datapath beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
` (7 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9f6fe1162..d0821ec3f3 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -686,6 +686,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 0f95d4ee85..3f0b851fbc 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -930,3 +1144,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 13/18] net/idpf: add support for write back based on ITR expire
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (11 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 12/18] net/idpf: support parsing packet type beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 14/18] net/idpf: add support for RSS beilei.xing
` (6 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 246 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d0821ec3f3..957cc10616 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -297,6 +297,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR here. The if
+ * condition should be updated when the FW can return the
+ * correct flag bits.
+ */
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -334,6 +418,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
int ret;
if (dev->data->mtu > vport->max_mtu) {
@@ -344,6 +432,27 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ ret = -EINVAL;
+ goto err_mtu;
+ }
+
+ ret = idpf_vc_alloc_vectors(vport, req_vecs_num);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ ret = idpf_config_rx_queues_irqs(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
@@ -376,6 +485,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -387,6 +500,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -748,6 +866,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 3f0b851fbc..827689f7f5 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -521,6 +525,8 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
+
args.ops = VIRTCHNL2_OP_GET_CAPS;
args.in_args = (uint8_t *)&caps_msg;
args.in_args_size = sizeof(caps_msg);
@@ -970,6 +976,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 14/18] net/idpf: add support for RSS
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (12 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 15/18] net/idpf: add support for Rx offloading beilei.xing
` (5 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 258 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 957cc10616..58560ea404 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
@@ -169,6 +171,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -189,6 +193,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -246,17 +254,110 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
return -ENOTSUP;
}
- if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
conf->rxmode.mq_mode);
return -ENOTSUP;
@@ -294,6 +395,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -ENOTSUP;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -500,6 +612,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 827689f7f5..9f72ae6264 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -525,6 +528,22 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
args.ops = VIRTCHNL2_OP_GET_CAPS;
@@ -615,6 +634,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 15/18] net/idpf: add support for Rx offloading
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (13 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 14/18] net/idpf: add support for RSS beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 16/18] net/idpf: add support for Tx offloading beilei.xing
` (4 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_vchnl.c | 18 +++++
4 files changed, 152 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d722c49fde..868571654f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,8 +3,13 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 58560ea404..a09f104425 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 9f72ae6264..495ee401c1 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -528,6 +528,24 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
caps_msg.rss_caps =
VIRTCHNL2_CAP_RSS_IPV4_TCP |
VIRTCHNL2_CAP_RSS_IPV4_UDP |
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 16/18] net/idpf: add support for Tx offloading
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (14 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 15/18] net/idpf: add support for Rx offloading beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
` (3 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Tx offloading support:
- support TSO for single queue model and split queue model.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 868571654f..d82b4aa0ff 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,6 +8,7 @@
;
[Features]
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index a09f104425..084426260c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -67,7 +67,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..cc296d7ab1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
}
return i;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (15 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 16/18] net/idpf: add support for Tx offloading beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 3:27 ` [PATCH v15 18/18] net/idpf: add support for timestamp offload beilei.xing
` (2 subsequent siblings)
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 084426260c..cd4ebcc2c6 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -69,7 +69,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cc296d7ab1..9e20f2b9d3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v15 18/18] net/idpf: add support for timestamp offload
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (16 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
@ 2022-10-29 3:27 ` beilei.xing
2022-10-29 14:48 ` [PATCH v15 00/18] add support for idpf PMD in DPDK Andrew Rybchenko
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
19 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-29 3:27 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjing Qiao
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d82b4aa0ff..099fd7f216 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -11,6 +11,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cd4ebcc2c6..50aac65daf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -65,7 +67,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9e20f2b9d3..bafa007faf 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v15 00/18] add support for idpf PMD in DPDK
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (17 preceding siblings ...)
2022-10-29 3:27 ` [PATCH v15 18/18] net/idpf: add support for timestamp offload beilei.xing
@ 2022-10-29 14:48 ` Andrew Rybchenko
2022-10-31 2:26 ` Xing, Beilei
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
19 siblings, 1 reply; 376+ messages in thread
From: Andrew Rybchenko @ 2022-10-29 14:48 UTC (permalink / raw)
To: beilei.xing, jingjing.wu; +Cc: dev, Thomas Monjalon
On 10/29/22 06:27, beilei.xing@intel.com wrote:
> From: Beilei Xing <beilei.xing@intel.com>
>
> This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
> The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
> Please refer to
> https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
> for more information.
>
> Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
>
> v2-v4:
> fixed some coding style issues and did some refactors.
>
> v5:
> fixed typo.
>
> v6-v9:
> fixed build errors and coding style issues.
>
> v11:
> - move shared code to common/idpf/base
> - Create one vport if there's no vport devargs
> - Refactor if conditions according to coding style
> - Refactor virtual channel return values
> - Refine dev_stop function
> - Refine RSS lut/key
> - Fix build error
>
> v12:
> - Refine dev_configure
> - Fix coding style according to the comments
> - Re-order patch
> - Romove dev_supported_ptypes_get
>
> v13:
> - refine dev_start/stop and queue_start/stop
> - fix timestamp offload
>
> v14:
> - fix wrong position for rte_validate_tx_offload
>
> v15:
> - refine the return value for ethdev ops.
> - removce forward static declarations.
> - refine get caps.
> - fix lock/unlock handling.
Applied to dpdk-next-net/main, thanks.
I've a number of concerns:
* conditional compilation IDPF_RX_PTYPE_OFFLOAD in [PATCH v15 17/18]
net/idpf: add AVX512 data path for single queue model
* the same prefix used for functions in common/idpf/base and net/idpf
drivers
* common/idpf/base uses own defines for negative errno (defined as a
number with corresponding errno in a comment). Strictly speaking it is
not the same, but work fine in a majority of cases
So, final decision will be done by Thomas on pulling to main tree.
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH v15 00/18] add support for idpf PMD in DPDK
2022-10-29 14:48 ` [PATCH v15 00/18] add support for idpf PMD in DPDK Andrew Rybchenko
@ 2022-10-31 2:26 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-10-31 2:26 UTC (permalink / raw)
To: Andrew Rybchenko, Wu, Jingjing; +Cc: dev, Thomas Monjalon
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Saturday, October 29, 2022 10:48 PM
> To: Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v15 00/18] add support for idpf PMD in DPDK
>
> On 10/29/22 06:27, beilei.xing@intel.com wrote:
> > From: Beilei Xing <beilei.xing@intel.com>
> >
> > This patchset introduced the idpf (Infrastructure Data Path Function) PMD
> in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
> > The Intel® IPU E2000 targets to deliver high performance under real
> workloads with security and isolation.
> > Please refer to
> > https://www.intel.com/content/www/us/en/products/network-
> io/infrastruc
> > ture-processing-units/asic/e2000-asic.html
> > for more information.
> >
> > Linux upstream is still ongoing, previous work refers to
> https://patchwork.ozlabs.org/project/intel-wired-
> lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
> >
> > v2-v4:
> > fixed some coding style issues and did some refactors.
> >
> > v5:
> > fixed typo.
> >
> > v6-v9:
> > fixed build errors and coding style issues.
> >
> > v11:
> > - move shared code to common/idpf/base
> > - Create one vport if there's no vport devargs
> > - Refactor if conditions according to coding style
> > - Refactor virtual channel return values
> > - Refine dev_stop function
> > - Refine RSS lut/key
> > - Fix build error
> >
> > v12:
> > - Refine dev_configure
> > - Fix coding style according to the comments
> > - Re-order patch
> > - Romove dev_supported_ptypes_get
> >
> > v13:
> > - refine dev_start/stop and queue_start/stop
> > - fix timestamp offload
> >
> > v14:
> > - fix wrong position for rte_validate_tx_offload
> >
> > v15:
> > - refine the return value for ethdev ops.
> > - removce forward static declarations.
> > - refine get caps.
> > - fix lock/unlock handling.
>
> Applied to dpdk-next-net/main, thanks.
>
> I've a number of concerns:
> * conditional compilation IDPF_RX_PTYPE_OFFLOAD in [PATCH v15 17/18]
Will remove the conditional compilation
> net/idpf: add AVX512 data path for single queue model
> * the same prefix used for functions in common/idpf/base and net/idpf
> drivers
I think the name of PMD and common library can be the same, right?
> * common/idpf/base uses own defines for negative errno (defined as a
> number with corresponding errno in a comment). Strictly speaking it is not
> the same, but work fine in a majority of cases
Make sense, will remove the own defines.
Thanks for your review, I saw the status in patchwork has been accepted, but didn't see idpf in dpdk-next-net, will send v16 to address the comments first.
>
> So, final decision will be done by Thomas on pulling to main tree.
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 00/18] add support for idpf PMD in DPDK
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
` (18 preceding siblings ...)
2022-10-29 14:48 ` [PATCH v15 00/18] add support for idpf PMD in DPDK Andrew Rybchenko
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 01/18] common/idpf: introduce common library beilei.xing
` (18 more replies)
19 siblings, 19 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev
From: Beilei Xing <beilei.xing@intel.com>
This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style according to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
v14:
- fix wrong position for rte_validate_tx_offload
v15:
- refine the return value for ethdev ops.
- removce forward static declarations.
- refine get caps.
- fix lock/unlock handling.
v16:
- refine errno in shared code
- remove the conditional compilation IDPF_RX_PTYPE_OFFLOAD
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 17 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1293 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1416 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15228 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 01/18] common/idpf: introduce common library
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 02/18] net/idpf: add support for device initialization beilei.xing
` (17 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiao Wang
From: Junfeng Guo <junfeng.guo@intel.com>
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8475 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..1debf129a3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = 0;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = -ENODEV;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = 0;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return -ENOMEM;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return 0;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return -ENOMEM;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ 0, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..8fcb3dcd93
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = 0;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return -EINVAL;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return -ENOMEM;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = -EBADR;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = -ENOMEM;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = 0;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = -ENOSPC;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*clean_count == 0)
+ return 0;
+ if (*clean_count > cq->ring_size)
+ return -EBADR;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = 0;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return -EBADR;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = 0;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*num_q_msg == 0)
+ return 0;
+ else if (*num_q_msg > cq->ring_size)
+ return -EBADR;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = -EBADMSG;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = -ENOMSG;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..2d79c7f4dd
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..3a272b1f8d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return 0;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return -ENOMEM;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return 0;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return -ENOMEM;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return -EINVAL;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return 0;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 02/18] net/idpf: add support for device initialization
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
2022-10-31 3:36 ` [PATCH v16 01/18] common/idpf: introduce common library beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 03/18] net/idpf: add Tx queue setup beilei.xing
` (16 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 891 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 416 ++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1662 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index bdf233c9f8..cc66db25e8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -770,6 +770,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 4d40ea29a3..12841ce407 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -34,6 +34,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 28812e092f..77674f2b06 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -158,6 +158,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..035f563275
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,891 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -EINVAL;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -ENOMEM;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -EINVAL;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -EINVAL;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -EINVAL;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ ret = -ENOMEM;
+ goto err_mbx;
+ }
+
+ ret = idpf_vc_check_api_version(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ ret = -ENOMEM;
+ goto err_api;
+ }
+
+ ret = idpf_vc_get_caps(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ ret = -ENOMEM;
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ ret = -ENOMEM;
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ ret = -ENOMEM;
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return ret;
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ int found = 0;
+
+ if (pci_dev == NULL)
+ return NULL;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ if (found == 0)
+ return NULL;
+
+ return adapter;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -ENOMEM;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..e89c2a2783
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != -ENOMSG)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -EINVAL;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -ENOMEM;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 589399400a..6470bf3636 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -29,6 +29,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 03/18] net/idpf: add Tx queue setup
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
2022-10-31 3:36 ` [PATCH v16 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 3:36 ` [PATCH v16 02/18] net/idpf: add support for device initialization beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 04/18] net/idpf: add Rx " beilei.xing
` (15 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/meson.build | 1 +
4 files changed, 448 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 035f563275..54f20d30ca 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -42,6 +43,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -631,6 +643,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 04/18] net/idpf: add Rx queue setup
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (2 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 03/18] net/idpf: add Tx queue setup beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 05/18] net/idpf: add support for device start and stop beilei.xing
` (14 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 54f20d30ca..fb5cd1b111 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -48,12 +48,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -643,6 +653,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 05/18] net/idpf: add support for device start and stop
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (3 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 04/18] net/idpf: add Rx " beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 06/18] net/idpf: add support for queue start beilei.xing
` (13 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index fb5cd1b111..621bf9aad5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,6 +29,22 @@ static const char * const idpf_valid_args[] = {
NULL
};
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static int
idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
{
@@ -267,6 +283,42 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ int ret;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -EINVAL;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ ret = idpf_vc_ena_dis_vport(vport, true);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
@@ -656,6 +708,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .link_update = idpf_dev_link_update,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 06/18] net/idpf: add support for queue start
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (4 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 05/18] net/idpf: add support for device start and stop beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 07/18] net/idpf: add support for queue stop beilei.xing
` (12 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 42 +++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 720 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 621bf9aad5..0400ed611f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -283,6 +283,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -296,11 +329,16 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ ret = idpf_start_queues(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return ret;
+ }
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
+ /* TODO: stop queues */
return ret;
}
@@ -711,6 +749,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.link_update = idpf_dev_link_update,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index e89c2a2783..08fb9061c2 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -390,6 +395,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 07/18] net/idpf: add support for queue stop
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (5 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 06/18] net/idpf: add support for queue start beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 08/18] net/idpf: add queue release beilei.xing
` (11 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 17 ++--
drivers/net/idpf/idpf_rxtx.c | 148 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
4 files changed, 242 insertions(+), 5 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0400ed611f..9f1e1e6a18 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -324,7 +324,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
@@ -332,17 +333,21 @@ idpf_dev_start(struct rte_eth_dev *dev)
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return ret;
+ goto err_mtu;
}
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- /* TODO: stop queues */
- return ret;
+ goto err_vport;
}
return 0;
+
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return ret;
}
static int
@@ -352,7 +357,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
@@ -751,6 +756,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.link_update = idpf_dev_link_update,
.rx_queue_start = idpf_rx_queue_start,
.tx_queue_start = idpf_tx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_stop = idpf_tx_queue_stop,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 08fb9061c2..f92141b54f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -837,6 +837,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 08/18] net/idpf: add queue release
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (6 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 07/18] net/idpf: add support for queue stop beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 09/18] net/idpf: add support for MTU configuration beilei.xing
` (10 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 9f1e1e6a18..1485f40e71 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -758,6 +758,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 09/18] net/idpf: add support for MTU configuration
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (7 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 08/18] net/idpf: add queue release beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 10/18] net/idpf: add support for basic Rx datapath beilei.xing
` (9 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 13 +++++++++++++
2 files changed, 14 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..d722c49fde 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1485f40e71..856f3d7266 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -760,6 +772,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .mtu_set = idpf_dev_mtu_set,
};
static uint16_t
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 10/18] net/idpf: add support for basic Rx datapath
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (8 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 09/18] net/idpf: add support for MTU configuration beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 11/18] net/idpf: add support for basic Tx datapath beilei.xing
` (8 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 856f3d7266..2f1f9eeee5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -348,6 +348,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 11/18] net/idpf: add support for basic Tx datapath
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (9 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 10/18] net/idpf: add support for basic Rx datapath beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 12/18] net/idpf: support parsing packet type beilei.xing
` (7 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 2f1f9eeee5..f9f6fe1162 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -349,6 +351,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 12/18] net/idpf: support parsing packet type
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (10 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 11/18] net/idpf: add support for basic Tx datapath beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
` (6 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9f6fe1162..d0821ec3f3 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -686,6 +686,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index f92141b54f..ced64dfaf6 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -930,3 +1144,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 13/18] net/idpf: add support for write back based on ITR expire
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (11 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 12/18] net/idpf: support parsing packet type beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 14/18] net/idpf: add support for RSS beilei.xing
` (5 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 246 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d0821ec3f3..957cc10616 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -297,6 +297,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR here. The if
+ * condition should be updated when the FW can return the
+ * correct flag bits.
+ */
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -334,6 +418,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
int ret;
if (dev->data->mtu > vport->max_mtu) {
@@ -344,6 +432,27 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ ret = -EINVAL;
+ goto err_mtu;
+ }
+
+ ret = idpf_vc_alloc_vectors(vport, req_vecs_num);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ ret = idpf_config_rx_queues_irqs(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
@@ -376,6 +485,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -387,6 +500,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -748,6 +866,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ced64dfaf6..5088d50b3e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -521,6 +525,8 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
+
args.ops = VIRTCHNL2_OP_GET_CAPS;
args.in_args = (uint8_t *)&caps_msg;
args.in_args_size = sizeof(caps_msg);
@@ -970,6 +976,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 14/18] net/idpf: add support for RSS
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (12 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 15/18] net/idpf: add support for Rx offloading beilei.xing
` (4 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 258 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 957cc10616..58560ea404 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
@@ -169,6 +171,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -189,6 +193,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -246,17 +254,110 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
return -ENOTSUP;
}
- if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
conf->rxmode.mq_mode);
return -ENOTSUP;
@@ -294,6 +395,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -ENOTSUP;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -500,6 +612,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 5088d50b3e..4ef354df89 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -525,6 +528,22 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
args.ops = VIRTCHNL2_OP_GET_CAPS;
@@ -615,6 +634,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 15/18] net/idpf: add support for Rx offloading
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (13 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 14/18] net/idpf: add support for RSS beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 16/18] net/idpf: add support for Tx offloading beilei.xing
` (3 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_vchnl.c | 18 +++++
4 files changed, 152 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d722c49fde..868571654f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,8 +3,13 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 58560ea404..a09f104425 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4ef354df89..00ac5b2a6b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -528,6 +528,24 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
caps_msg.rss_caps =
VIRTCHNL2_CAP_RSS_IPV4_TCP |
VIRTCHNL2_CAP_RSS_IPV4_UDP |
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 16/18] net/idpf: add support for Tx offloading
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (14 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 15/18] net/idpf: add support for Rx offloading beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
` (2 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Tx offloading support:
- support TSO for single queue model and split queue model.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 868571654f..d82b4aa0ff 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,6 +8,7 @@
;
[Features]
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index a09f104425..084426260c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -67,7 +67,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..cc296d7ab1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
}
return i;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (15 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 16/18] net/idpf: add support for Tx offloading beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 3:36 ` [PATCH v16 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 084426260c..cd4ebcc2c6 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -69,7 +69,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cc296d7ab1..9e20f2b9d3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v16 18/18] net/idpf: add support for timestamp offload
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (16 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
@ 2022-10-31 3:36 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 3:36 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjing Qiao
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d82b4aa0ff..099fd7f216 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -11,6 +11,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cd4ebcc2c6..50aac65daf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -65,7 +67,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9e20f2b9d3..bafa007faf 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 00/18] add support for idpf PMD in DPDK
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
` (17 preceding siblings ...)
2022-10-31 3:36 ` [PATCH v16 18/18] net/idpf: add support for timestamp offload beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 01/18] common/idpf: introduce common library beilei.xing
` (18 more replies)
18 siblings, 19 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev
From: Beilei Xing <beilei.xing@intel.com>
This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style according to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
v14:
- fix wrong position for rte_validate_tx_offload
v15:
- refine the return value for ethdev ops.
- removce forward static declarations.
- refine get caps.
- fix lock/unlock handling.
v16:
- refine errno in shared code
- remove the conditional compilation IDPF_RX_PTYPE_OFFLOAD
v17:
- fix build error on FreeBSD
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 17 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1293 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1416 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15228 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 01/18] common/idpf: introduce common library
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 02/18] net/idpf: add support for device initialization beilei.xing
` (17 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiao Wang
From: Junfeng Guo <junfeng.guo@intel.com>
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8475 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..1debf129a3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = 0;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = -ENODEV;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = 0;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return -ENOMEM;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return 0;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return -ENOMEM;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ 0, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..3af81e5a64
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = 0;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return -EINVAL;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return -ENOMEM;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = -ENOMEM;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = 0;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = -ENOSPC;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*clean_count == 0)
+ return 0;
+ if (*clean_count > cq->ring_size)
+ return -EINVAL;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = 0;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return -EINVAL;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = 0;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*num_q_msg == 0)
+ return 0;
+ else if (*num_q_msg > cq->ring_size)
+ return -EINVAL;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = -EBADMSG;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = -ENOMSG;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..2d79c7f4dd
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..3a272b1f8d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return 0;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return -ENOMEM;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return 0;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return -ENOMEM;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return -EINVAL;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return 0;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 02/18] net/idpf: add support for device initialization
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 5:15 ` [PATCH v17 01/18] common/idpf: introduce common library beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 03/18] net/idpf: add Tx queue setup beilei.xing
` (16 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 891 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 416 ++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1662 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index bdf233c9f8..cc66db25e8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -770,6 +770,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 4d40ea29a3..12841ce407 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -34,6 +34,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 28812e092f..77674f2b06 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -158,6 +158,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..035f563275
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,891 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -EINVAL;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -ENOMEM;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -EINVAL;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -EINVAL;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -EINVAL;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ ret = -ENOMEM;
+ goto err_mbx;
+ }
+
+ ret = idpf_vc_check_api_version(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ ret = -ENOMEM;
+ goto err_api;
+ }
+
+ ret = idpf_vc_get_caps(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ ret = -ENOMEM;
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ ret = -ENOMEM;
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ ret = -ENOMEM;
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return ret;
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ int found = 0;
+
+ if (pci_dev == NULL)
+ return NULL;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ if (found == 0)
+ return NULL;
+
+ return adapter;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -ENOMEM;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..e89c2a2783
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != -ENOMSG)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -EINVAL;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -ENOMEM;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 589399400a..6470bf3636 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -29,6 +29,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 03/18] net/idpf: add Tx queue setup
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 5:15 ` [PATCH v17 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 5:15 ` [PATCH v17 02/18] net/idpf: add support for device initialization beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 04/18] net/idpf: add Rx " beilei.xing
` (15 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/meson.build | 1 +
4 files changed, 448 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 035f563275..54f20d30ca 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -42,6 +43,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -631,6 +643,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 04/18] net/idpf: add Rx queue setup
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (2 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 03/18] net/idpf: add Tx queue setup beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 05/18] net/idpf: add support for device start and stop beilei.xing
` (14 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 54f20d30ca..fb5cd1b111 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -48,12 +48,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -643,6 +653,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 05/18] net/idpf: add support for device start and stop
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (3 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 04/18] net/idpf: add Rx " beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 06/18] net/idpf: add support for queue start beilei.xing
` (13 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index fb5cd1b111..621bf9aad5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,6 +29,22 @@ static const char * const idpf_valid_args[] = {
NULL
};
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static int
idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
{
@@ -267,6 +283,42 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ int ret;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -EINVAL;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ ret = idpf_vc_ena_dis_vport(vport, true);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
@@ -656,6 +708,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .link_update = idpf_dev_link_update,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 06/18] net/idpf: add support for queue start
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (4 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 05/18] net/idpf: add support for device start and stop beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 07/18] net/idpf: add support for queue stop beilei.xing
` (12 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 42 +++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 720 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 621bf9aad5..0400ed611f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -283,6 +283,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -296,11 +329,16 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ ret = idpf_start_queues(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return ret;
+ }
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
+ /* TODO: stop queues */
return ret;
}
@@ -711,6 +749,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.link_update = idpf_dev_link_update,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index e89c2a2783..08fb9061c2 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -390,6 +395,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 07/18] net/idpf: add support for queue stop
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (5 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 06/18] net/idpf: add support for queue start beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 08/18] net/idpf: add queue release beilei.xing
` (11 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 17 ++--
drivers/net/idpf/idpf_rxtx.c | 148 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
4 files changed, 242 insertions(+), 5 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0400ed611f..9f1e1e6a18 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -324,7 +324,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
@@ -332,17 +333,21 @@ idpf_dev_start(struct rte_eth_dev *dev)
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return ret;
+ goto err_mtu;
}
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- /* TODO: stop queues */
- return ret;
+ goto err_vport;
}
return 0;
+
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return ret;
}
static int
@@ -352,7 +357,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
@@ -751,6 +756,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.link_update = idpf_dev_link_update,
.rx_queue_start = idpf_rx_queue_start,
.tx_queue_start = idpf_tx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_stop = idpf_tx_queue_stop,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 08fb9061c2..f92141b54f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -837,6 +837,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 08/18] net/idpf: add queue release
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (6 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 07/18] net/idpf: add support for queue stop beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 09/18] net/idpf: add support for MTU configuration beilei.xing
` (10 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 9f1e1e6a18..1485f40e71 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -758,6 +758,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 09/18] net/idpf: add support for MTU configuration
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (7 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 08/18] net/idpf: add queue release beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 10/18] net/idpf: add support for basic Rx datapath beilei.xing
` (9 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 13 +++++++++++++
2 files changed, 14 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..d722c49fde 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1485f40e71..856f3d7266 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -760,6 +772,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .mtu_set = idpf_dev_mtu_set,
};
static uint16_t
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 10/18] net/idpf: add support for basic Rx datapath
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (8 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 09/18] net/idpf: add support for MTU configuration beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 11/18] net/idpf: add support for basic Tx datapath beilei.xing
` (8 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 856f3d7266..2f1f9eeee5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -348,6 +348,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 11/18] net/idpf: add support for basic Tx datapath
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (9 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 10/18] net/idpf: add support for basic Rx datapath beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 12/18] net/idpf: support parsing packet type beilei.xing
` (7 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 2f1f9eeee5..f9f6fe1162 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -349,6 +351,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 12/18] net/idpf: support parsing packet type
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (10 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 11/18] net/idpf: add support for basic Tx datapath beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
` (6 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9f6fe1162..d0821ec3f3 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -686,6 +686,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index f92141b54f..ced64dfaf6 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -930,3 +1144,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 13/18] net/idpf: add support for write back based on ITR expire
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (11 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 12/18] net/idpf: support parsing packet type beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 14/18] net/idpf: add support for RSS beilei.xing
` (5 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 246 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d0821ec3f3..957cc10616 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -297,6 +297,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR here. The if
+ * condition should be updated when the FW can return the
+ * correct flag bits.
+ */
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -334,6 +418,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
int ret;
if (dev->data->mtu > vport->max_mtu) {
@@ -344,6 +432,27 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ ret = -EINVAL;
+ goto err_mtu;
+ }
+
+ ret = idpf_vc_alloc_vectors(vport, req_vecs_num);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ ret = idpf_config_rx_queues_irqs(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
@@ -376,6 +485,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -387,6 +500,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -748,6 +866,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ced64dfaf6..5088d50b3e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -521,6 +525,8 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
+
args.ops = VIRTCHNL2_OP_GET_CAPS;
args.in_args = (uint8_t *)&caps_msg;
args.in_args_size = sizeof(caps_msg);
@@ -970,6 +976,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 14/18] net/idpf: add support for RSS
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (12 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 15/18] net/idpf: add support for Rx offloading beilei.xing
` (4 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 258 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 957cc10616..58560ea404 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
@@ -169,6 +171,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -189,6 +193,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -246,17 +254,110 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
return -ENOTSUP;
}
- if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
conf->rxmode.mq_mode);
return -ENOTSUP;
@@ -294,6 +395,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -ENOTSUP;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -500,6 +612,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 5088d50b3e..4ef354df89 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -525,6 +528,22 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
args.ops = VIRTCHNL2_OP_GET_CAPS;
@@ -615,6 +634,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 15/18] net/idpf: add support for Rx offloading
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (13 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 14/18] net/idpf: add support for RSS beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 16/18] net/idpf: add support for Tx offloading beilei.xing
` (3 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_vchnl.c | 18 +++++
4 files changed, 152 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d722c49fde..868571654f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,8 +3,13 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 58560ea404..a09f104425 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4ef354df89..00ac5b2a6b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -528,6 +528,24 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
caps_msg.rss_caps =
VIRTCHNL2_CAP_RSS_IPV4_TCP |
VIRTCHNL2_CAP_RSS_IPV4_UDP |
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 16/18] net/idpf: add support for Tx offloading
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (14 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 15/18] net/idpf: add support for Rx offloading beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
` (2 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Tx offloading support:
- support TSO for single queue model and split queue model.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 868571654f..d82b4aa0ff 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,6 +8,7 @@
;
[Features]
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index a09f104425..084426260c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -67,7 +67,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..cc296d7ab1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
}
return i;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (15 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 16/18] net/idpf: add support for Tx offloading beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 5:15 ` [PATCH v17 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 871 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1191 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 084426260c..cd4ebcc2c6 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -69,7 +69,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cc296d7ab1..9e20f2b9d3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..822b79f4d8
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,871 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+/******************************************************************************
+ * If user knows a specific offload is not enabled by APP,
+ * the macro can be commented to save the effort of fast path.
+ * Currently below 1 feature is supported in RX path,
+ * 1, packet type analysis
+ ******************************************************************************/
+#define IDPF_RX_PTYPE_OFFLOAD
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+#endif
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+#endif
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+#ifdef IDPF_RX_PTYPE_OFFLOAD
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+#endif
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v17 18/18] net/idpf: add support for timestamp offload
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (16 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
@ 2022-10-31 5:15 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 5:15 UTC (permalink / raw)
To: andrew.rybchenko, jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjing Qiao
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d82b4aa0ff..099fd7f216 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -11,6 +11,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cd4ebcc2c6..50aac65daf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -65,7 +67,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9e20f2b9d3..bafa007faf 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 00/18] add support for idpf PMD in DPDK
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
` (17 preceding siblings ...)
2022-10-31 5:15 ` [PATCH v17 18/18] net/idpf: add support for timestamp offload beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 01/18] common/idpf: introduce common library beilei.xing
` (18 more replies)
18 siblings, 19 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev
From: Beilei Xing <beilei.xing@intel.com>
This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
Please refer to
https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
for more information.
Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
v2-v4:
fixed some coding style issues and did some refactors.
v5:
fixed typo.
v6-v9:
fixed build errors and coding style issues.
v11:
- move shared code to common/idpf/base
- Create one vport if there's no vport devargs
- Refactor if conditions according to coding style
- Refactor virtual channel return values
- Refine dev_stop function
- Refine RSS lut/key
- Fix build error
v12:
- Refine dev_configure
- Fix coding style according to the comments
- Re-order patch
- Romove dev_supported_ptypes_get
v13:
- refine dev_start/stop and queue_start/stop
- fix timestamp offload
v14:
- fix wrong position for rte_validate_tx_offload
v15:
- refine the return value for ethdev ops.
- removce forward static declarations.
- refine get caps.
- fix lock/unlock handling.
v16:
- refine errno in shared code
- remove the conditional compilation IDPF_RX_PTYPE_OFFLOAD
v17:
- fix build error on FreeBSD
v18:
- remove the conditional compilation IDPF_RX_PTYPE_OFFLOAD
Junfeng Guo (18):
common/idpf: introduce common library
net/idpf: add support for device initialization
net/idpf: add Tx queue setup
net/idpf: add Rx queue setup
net/idpf: add support for device start and stop
net/idpf: add support for queue start
net/idpf: add support for queue stop
net/idpf: add queue release
net/idpf: add support for MTU configuration
net/idpf: add support for basic Rx datapath
net/idpf: add support for basic Tx datapath
net/idpf: support parsing packet type
net/idpf: add support for write back based on ITR expire
net/idpf: add support for RSS
net/idpf: add support for Rx offloading
net/idpf: add support for Tx offloading
net/idpf: add AVX512 data path for single queue model
net/idpf: add support for timestamp offload
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 17 +
doc/guides/nics/idpf.rst | 85 +
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
drivers/net/idpf/idpf_ethdev.c | 1293 ++++++++
drivers/net/idpf/idpf_ethdev.h | 252 ++
drivers/net/idpf/idpf_logs.h | 56 +
drivers/net/idpf/idpf_rxtx.c | 2308 +++++++++++++
drivers/net/idpf/idpf_rxtx.h | 291 ++
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 857 +++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +
drivers/net/idpf/idpf_vchnl.c | 1416 ++++++++
drivers/net/idpf/meson.build | 44 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
38 files changed, 15214 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 01/18] common/idpf: introduce common library
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 02/18] net/idpf: add support for device initialization beilei.xing
` (17 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiao Wang
From: Junfeng Guo <junfeng.guo@intel.com>
Introduce common library for IDPF (Infrastructure Data
Path Function) PMD.
Add base code and OS specific implementation first.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/common/idpf/base/idpf_alloc.h | 22 +
drivers/common/idpf/base/idpf_common.c | 364 +++
drivers/common/idpf/base/idpf_controlq.c | 691 ++++
drivers/common/idpf/base/idpf_controlq.h | 224 ++
drivers/common/idpf/base/idpf_controlq_api.h | 207 ++
.../common/idpf/base/idpf_controlq_setup.c | 179 +
drivers/common/idpf/base/idpf_devids.h | 18 +
drivers/common/idpf/base/idpf_lan_pf_regs.h | 134 +
drivers/common/idpf/base/idpf_lan_txrx.h | 428 +++
drivers/common/idpf/base/idpf_lan_vf_regs.h | 114 +
drivers/common/idpf/base/idpf_osdep.h | 364 +++
drivers/common/idpf/base/idpf_prototype.h | 45 +
drivers/common/idpf/base/idpf_type.h | 106 +
drivers/common/idpf/base/meson.build | 14 +
drivers/common/idpf/base/siov_regs.h | 47 +
drivers/common/idpf/base/virtchnl.h | 2866 +++++++++++++++++
drivers/common/idpf/base/virtchnl2.h | 1462 +++++++++
drivers/common/idpf/base/virtchnl2_lan_desc.h | 606 ++++
.../common/idpf/base/virtchnl_inline_ipsec.h | 567 ++++
drivers/common/idpf/meson.build | 4 +
drivers/common/idpf/version.map | 12 +
drivers/common/meson.build | 1 +
22 files changed, 8475 insertions(+)
create mode 100644 drivers/common/idpf/base/idpf_alloc.h
create mode 100644 drivers/common/idpf/base/idpf_common.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.c
create mode 100644 drivers/common/idpf/base/idpf_controlq.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_api.h
create mode 100644 drivers/common/idpf/base/idpf_controlq_setup.c
create mode 100644 drivers/common/idpf/base/idpf_devids.h
create mode 100644 drivers/common/idpf/base/idpf_lan_pf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_lan_txrx.h
create mode 100644 drivers/common/idpf/base/idpf_lan_vf_regs.h
create mode 100644 drivers/common/idpf/base/idpf_osdep.h
create mode 100644 drivers/common/idpf/base/idpf_prototype.h
create mode 100644 drivers/common/idpf/base/idpf_type.h
create mode 100644 drivers/common/idpf/base/meson.build
create mode 100644 drivers/common/idpf/base/siov_regs.h
create mode 100644 drivers/common/idpf/base/virtchnl.h
create mode 100644 drivers/common/idpf/base/virtchnl2.h
create mode 100644 drivers/common/idpf/base/virtchnl2_lan_desc.h
create mode 100644 drivers/common/idpf/base/virtchnl_inline_ipsec.h
create mode 100644 drivers/common/idpf/meson.build
create mode 100644 drivers/common/idpf/version.map
diff --git a/drivers/common/idpf/base/idpf_alloc.h b/drivers/common/idpf/base/idpf_alloc.h
new file mode 100644
index 0000000000..bc054851b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_alloc.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ALLOC_H_
+#define _IDPF_ALLOC_H_
+
+/* Memory types */
+enum idpf_memset_type {
+ IDPF_NONDMA_MEM = 0,
+ IDPF_DMA_MEM
+};
+
+/* Memcpy types */
+enum idpf_memcpy_type {
+ IDPF_NONDMA_TO_NONDMA = 0,
+ IDPF_NONDMA_TO_DMA,
+ IDPF_DMA_TO_DMA,
+ IDPF_DMA_TO_NONDMA
+};
+
+#endif /* _IDPF_ALLOC_H_ */
diff --git a/drivers/common/idpf/base/idpf_common.c b/drivers/common/idpf/base/idpf_common.c
new file mode 100644
index 0000000000..1debf129a3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_common.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_type.h"
+#include "idpf_prototype.h"
+#include "virtchnl.h"
+
+
+/**
+ * idpf_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the mac type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+int idpf_set_mac_type(struct idpf_hw *hw)
+{
+ int status = 0;
+
+ DEBUGFUNC("Set MAC type\n");
+
+ if (hw->vendor_id == IDPF_INTEL_VENDOR_ID) {
+ switch (hw->device_id) {
+ case IDPF_DEV_ID_PF:
+ hw->mac.type = IDPF_MAC_PF;
+ break;
+ case IDPF_DEV_ID_VF:
+ hw->mac.type = IDPF_MAC_VF;
+ break;
+ default:
+ hw->mac.type = IDPF_MAC_GENERIC;
+ break;
+ }
+ } else {
+ status = -ENODEV;
+ }
+
+ DEBUGOUT2("Setting MAC type found mac: %d, returns: %d\n",
+ hw->mac.type, status);
+ return status;
+}
+
+/**
+ * idpf_init_hw - main initialization routine
+ * @hw: pointer to the hardware structure
+ * @ctlq_size: struct to pass ctlq size data
+ */
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size)
+{
+ struct idpf_ctlq_create_info *q_info;
+ int status = 0;
+ struct idpf_ctlq_info *cq = NULL;
+
+ /* Setup initial control queues */
+ q_info = (struct idpf_ctlq_create_info *)
+ idpf_calloc(hw, 2, sizeof(struct idpf_ctlq_create_info));
+ if (!q_info)
+ return -ENOMEM;
+
+ q_info[0].type = IDPF_CTLQ_TYPE_MAILBOX_TX;
+ q_info[0].buf_size = ctlq_size.asq_buf_size;
+ q_info[0].len = ctlq_size.asq_ring_size;
+ q_info[0].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[0].reg.head = PF_FW_ATQH;
+ q_info[0].reg.tail = PF_FW_ATQT;
+ q_info[0].reg.len = PF_FW_ATQLEN;
+ q_info[0].reg.bah = PF_FW_ATQBAH;
+ q_info[0].reg.bal = PF_FW_ATQBAL;
+ q_info[0].reg.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = PF_FW_ATQH_ATQH_M;
+ } else {
+ q_info[0].reg.head = VF_ATQH;
+ q_info[0].reg.tail = VF_ATQT;
+ q_info[0].reg.len = VF_ATQLEN;
+ q_info[0].reg.bah = VF_ATQBAH;
+ q_info[0].reg.bal = VF_ATQBAL;
+ q_info[0].reg.len_mask = VF_ATQLEN_ATQLEN_M;
+ q_info[0].reg.len_ena_mask = VF_ATQLEN_ATQENABLE_M;
+ q_info[0].reg.head_mask = VF_ATQH_ATQH_M;
+ }
+
+ q_info[1].type = IDPF_CTLQ_TYPE_MAILBOX_RX;
+ q_info[1].buf_size = ctlq_size.arq_buf_size;
+ q_info[1].len = ctlq_size.arq_ring_size;
+ q_info[1].id = -1; /* default queue */
+
+ if (hw->mac.type == IDPF_MAC_PF) {
+ q_info[1].reg.head = PF_FW_ARQH;
+ q_info[1].reg.tail = PF_FW_ARQT;
+ q_info[1].reg.len = PF_FW_ARQLEN;
+ q_info[1].reg.bah = PF_FW_ARQBAH;
+ q_info[1].reg.bal = PF_FW_ARQBAL;
+ q_info[1].reg.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = PF_FW_ARQH_ARQH_M;
+ } else {
+ q_info[1].reg.head = VF_ARQH;
+ q_info[1].reg.tail = VF_ARQT;
+ q_info[1].reg.len = VF_ARQLEN;
+ q_info[1].reg.bah = VF_ARQBAH;
+ q_info[1].reg.bal = VF_ARQBAL;
+ q_info[1].reg.len_mask = VF_ARQLEN_ARQLEN_M;
+ q_info[1].reg.len_ena_mask = VF_ARQLEN_ARQENABLE_M;
+ q_info[1].reg.head_mask = VF_ARQH_ARQH_M;
+ }
+
+ status = idpf_ctlq_init(hw, 2, q_info);
+ if (status) {
+ /* TODO return error */
+ idpf_free(hw, q_info);
+ return status;
+ }
+
+ LIST_FOR_EACH_ENTRY(cq, &hw->cq_list_head, idpf_ctlq_info, cq_list) {
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = cq;
+ else if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = cq;
+ }
+
+ /* TODO hardcode a mac addr for now */
+ hw->mac.addr[0] = 0x00;
+ hw->mac.addr[1] = 0x00;
+ hw->mac.addr[2] = 0x00;
+ hw->mac.addr[3] = 0x00;
+ hw->mac.addr[4] = 0x03;
+ hw->mac.addr[5] = 0x14;
+
+ return 0;
+}
+
+/**
+ * idpf_send_msg_to_cp
+ * @hw: pointer to the hardware structure
+ * @v_opcode: opcodes for VF-PF communication
+ * @v_retval: return error code
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ * @cmd_details: pointer to command details
+ *
+ * Send message to CP. By default, this message
+ * is sent asynchronously, i.e. idpf_asq_send_command() does not wait for
+ * completion before returning.
+ */
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen)
+{
+ struct idpf_ctlq_msg ctlq_msg = { 0 };
+ struct idpf_dma_mem dma_mem = { 0 };
+ int status;
+
+ ctlq_msg.opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg.func_id = 0;
+ ctlq_msg.data_len = msglen;
+ ctlq_msg.cookie.mbx.chnl_retval = v_retval;
+ ctlq_msg.cookie.mbx.chnl_opcode = v_opcode;
+
+ if (msglen > 0) {
+ dma_mem.va = (struct idpf_dma_mem *)
+ idpf_alloc_dma_mem(hw, &dma_mem, msglen);
+ if (!dma_mem.va)
+ return -ENOMEM;
+
+ idpf_memcpy(dma_mem.va, msg, msglen, IDPF_NONDMA_TO_DMA);
+ ctlq_msg.ctx.indirect.payload = &dma_mem;
+ }
+ status = idpf_ctlq_send(hw, hw->asq, 1, &ctlq_msg);
+
+ if (dma_mem.va)
+ idpf_free_dma_mem(hw, &dma_mem);
+
+ return status;
+}
+
+/**
+ * idpf_asq_done - check if FW has processed the Admin Send Queue
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+bool idpf_asq_done(struct idpf_hw *hw)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, hw->asq->reg.head) == hw->asq->next_to_use;
+}
+
+/**
+ * idpf_check_asq_alive
+ * @hw: pointer to the hw struct
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool idpf_check_asq_alive(struct idpf_hw *hw)
+{
+ if (hw->asq->reg.len)
+ return !!(rd32(hw, hw->asq->reg.len) &
+ PF_FW_ATQLEN_ATQENABLE_M);
+
+ return false;
+}
+
+/**
+ * idpf_clean_arq_element
+ * @hw: pointer to the hw struct
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'
+ */
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e, u16 *pending)
+{
+ struct idpf_ctlq_msg msg = { 0 };
+ int status;
+
+ *pending = 1;
+
+ status = idpf_ctlq_recv(hw->arq, pending, &msg);
+
+ /* ctlq_msg does not align to ctlq_desc, so copy relevant data here */
+ e->desc.opcode = msg.opcode;
+ e->desc.cookie_high = msg.cookie.mbx.chnl_opcode;
+ e->desc.cookie_low = msg.cookie.mbx.chnl_retval;
+ e->desc.ret_val = msg.status;
+ e->desc.datalen = msg.data_len;
+ if (msg.data_len > 0) {
+ e->buf_len = msg.data_len;
+ idpf_memcpy(e->msg_buf, msg.ctx.indirect.payload->va, msg.data_len,
+ IDPF_DMA_TO_NONDMA);
+ }
+ return status;
+}
+
+/**
+ * idpf_deinit_hw - shutdown routine
+ * @hw: pointer to the hardware structure
+ */
+int idpf_deinit_hw(struct idpf_hw *hw)
+{
+ hw->asq = NULL;
+ hw->arq = NULL;
+
+ return idpf_ctlq_deinit(hw);
+}
+
+/**
+ * idpf_reset
+ * @hw: pointer to the hardware structure
+ *
+ * Send a RESET message to the CPF. Does not wait for response from CPF
+ * as none will be forthcoming. Immediately after calling this function,
+ * the control queue should be shut down and (optionally) reinitialized.
+ */
+int idpf_reset(struct idpf_hw *hw)
+{
+ return idpf_send_msg_to_cp(hw, VIRTCHNL_OP_RESET_VF,
+ 0, NULL, 0);
+}
+
+/**
+ * idpf_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get or set RSS look up table
+ */
+STATIC int idpf_get_set_rss_lut(struct idpf_hw *hw, u16 vsi_id,
+ bool pf_lut, u8 *lut, u16 lut_size,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, false);
+}
+
+/**
+ * idpf_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: vsi fw index
+ * @pf_lut: for PF table set true, for VSI table set false
+ * @lut: pointer to the lut buffer provided by the caller
+ * @lut_size: size of the lut buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 vsi_id, bool pf_lut,
+ u8 *lut, u16 lut_size)
+{
+ return idpf_get_set_rss_lut(hw, vsi_id, pf_lut, lut, lut_size, true);
+}
+
+/**
+ * idpf_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get the RSS key per VSI
+ */
+STATIC int idpf_get_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key,
+ bool set)
+{
+ /* TODO fill out command */
+ return 0;
+}
+
+/**
+ * idpf_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ */
+int idpf_get_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * idpf_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: vsi fw index
+ * @key: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+int idpf_set_rss_key(struct idpf_hw *hw, u16 vsi_id,
+ struct idpf_get_set_rss_key_data *key)
+{
+ return idpf_get_set_rss_key(hw, vsi_id, key, true);
+}
+
+RTE_LOG_REGISTER_DEFAULT(idpf_common_logger, NOTICE);
diff --git a/drivers/common/idpf/base/idpf_controlq.c b/drivers/common/idpf/base/idpf_controlq.c
new file mode 100644
index 0000000000..3af81e5a64
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.c
@@ -0,0 +1,691 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#include "idpf_controlq.h"
+
+/**
+ * idpf_ctlq_setup_regs - initialize control queue registers
+ * @cq: pointer to the specific control queue
+ * @q_create_info: structs containing info for each queue to be initialized
+ */
+static void
+idpf_ctlq_setup_regs(struct idpf_ctlq_info *cq,
+ struct idpf_ctlq_create_info *q_create_info)
+{
+ /* set head and tail registers in our local struct */
+ cq->reg.head = q_create_info->reg.head;
+ cq->reg.tail = q_create_info->reg.tail;
+ cq->reg.len = q_create_info->reg.len;
+ cq->reg.bah = q_create_info->reg.bah;
+ cq->reg.bal = q_create_info->reg.bal;
+ cq->reg.len_mask = q_create_info->reg.len_mask;
+ cq->reg.len_ena_mask = q_create_info->reg.len_ena_mask;
+ cq->reg.head_mask = q_create_info->reg.head_mask;
+}
+
+/**
+ * idpf_ctlq_init_regs - Initialize control queue registers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ * @is_rxq: true if receive control queue, false otherwise
+ *
+ * Initialize registers. The caller is expected to have already initialized the
+ * descriptor ring memory and buffer memory
+ */
+static void idpf_ctlq_init_regs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ bool is_rxq)
+{
+ /* Update tail to post pre-allocated buffers for rx queues */
+ if (is_rxq)
+ wr32(hw, cq->reg.tail, (u32)(cq->ring_size - 1));
+
+ /* For non-Mailbox control queues only TAIL need to be set */
+ if (cq->q_id != -1)
+ return;
+
+ /* Clear Head for both send or receive */
+ wr32(hw, cq->reg.head, 0);
+
+ /* set starting point */
+ wr32(hw, cq->reg.bal, IDPF_LO_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.bah, IDPF_HI_DWORD(cq->desc_ring.pa));
+ wr32(hw, cq->reg.len, (cq->ring_size | cq->reg.len_ena_mask));
+}
+
+/**
+ * idpf_ctlq_init_rxq_bufs - populate receive queue descriptors with buf
+ * @cq: pointer to the specific Control queue
+ *
+ * Record the address of the receive queue DMA buffers in the descriptors.
+ * The buffers must have been previously allocated.
+ */
+static void idpf_ctlq_init_rxq_bufs(struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ for (i = 0; i < cq->ring_size; i++) {
+ struct idpf_ctlq_desc *desc = IDPF_CTLQ_DESC(cq, i);
+ struct idpf_dma_mem *bi = cq->bi.rx_buff[i];
+
+ /* No buffer to post to descriptor, continue */
+ if (!bi)
+ continue;
+
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+ desc->opcode = 0;
+ desc->datalen = (__le16)CPU_TO_LE16(bi->size);
+ desc->ret_val = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(bi->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(bi->pa));
+ desc->params.indirect.param0 = 0;
+ desc->params.indirect.param1 = 0;
+ }
+}
+
+/**
+ * idpf_ctlq_shutdown - shutdown the CQ
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for any controq queue
+ */
+static void idpf_ctlq_shutdown(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (!cq->ring_size)
+ goto shutdown_sq_out;
+
+#ifdef SIMICS_BUILD
+ wr32(hw, cq->reg.head, 0);
+ wr32(hw, cq->reg.tail, 0);
+ wr32(hw, cq->reg.len, 0);
+ wr32(hw, cq->reg.bal, 0);
+ wr32(hw, cq->reg.bah, 0);
+#endif /* SIMICS_BUILD */
+
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, cq);
+
+ /* Set ring_size to 0 to indicate uninitialized queue */
+ cq->ring_size = 0;
+
+shutdown_sq_out:
+ idpf_release_lock(&cq->cq_lock);
+ idpf_destroy_lock(&cq->cq_lock);
+}
+
+/**
+ * idpf_ctlq_add - add one control queue
+ * @hw: pointer to hardware struct
+ * @qinfo: info for queue to be created
+ * @cq_out: (output) double pointer to control queue to be created
+ *
+ * Allocate and initialize a control queue and add it to the control queue list.
+ * The cq parameter will be allocated/initialized and passed back to the caller
+ * if no errors occur.
+ *
+ * Note: idpf_ctlq_init must be called prior to any calls to idpf_ctlq_add
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq_out)
+{
+ bool is_rxq = false;
+ int status = 0;
+
+ if (!qinfo->len || !qinfo->buf_size ||
+ qinfo->len > IDPF_CTLQ_MAX_RING_SIZE ||
+ qinfo->buf_size > IDPF_CTLQ_MAX_BUF_LEN)
+ return -EINVAL;
+
+ *cq_out = (struct idpf_ctlq_info *)
+ idpf_calloc(hw, 1, sizeof(struct idpf_ctlq_info));
+ if (!(*cq_out))
+ return -ENOMEM;
+
+ (*cq_out)->cq_type = qinfo->type;
+ (*cq_out)->q_id = qinfo->id;
+ (*cq_out)->buf_size = qinfo->buf_size;
+ (*cq_out)->ring_size = qinfo->len;
+
+ (*cq_out)->next_to_use = 0;
+ (*cq_out)->next_to_clean = 0;
+ (*cq_out)->next_to_post = (*cq_out)->ring_size - 1;
+
+ switch (qinfo->type) {
+ case IDPF_CTLQ_TYPE_MAILBOX_RX:
+ is_rxq = true;
+#ifdef __KERNEL__
+ fallthrough;
+#else
+ /* fallthrough */
+#endif /* __KERNEL__ */
+ case IDPF_CTLQ_TYPE_MAILBOX_TX:
+ status = idpf_ctlq_alloc_ring_res(hw, *cq_out);
+ break;
+ default:
+ status = -EINVAL;
+ break;
+ }
+
+ if (status)
+ goto init_free_q;
+
+ if (is_rxq) {
+ idpf_ctlq_init_rxq_bufs(*cq_out);
+ } else {
+ /* Allocate the array of msg pointers for TX queues */
+ (*cq_out)->bi.tx_msg = (struct idpf_ctlq_msg **)
+ idpf_calloc(hw, qinfo->len,
+ sizeof(struct idpf_ctlq_msg *));
+ if (!(*cq_out)->bi.tx_msg) {
+ status = -ENOMEM;
+ goto init_dealloc_q_mem;
+ }
+ }
+
+ idpf_ctlq_setup_regs(*cq_out, qinfo);
+
+ idpf_ctlq_init_regs(hw, *cq_out, is_rxq);
+
+ idpf_init_lock(&(*cq_out)->cq_lock);
+
+ LIST_INSERT_HEAD(&hw->cq_list_head, (*cq_out), cq_list);
+
+ return status;
+
+init_dealloc_q_mem:
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_dealloc_ring_res(hw, *cq_out);
+init_free_q:
+ idpf_free(hw, *cq_out);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_remove - deallocate and remove specified control queue
+ * @hw: pointer to hardware struct
+ * @cq: pointer to control queue to be removed
+ */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ LIST_REMOVE(cq, cq_list);
+ idpf_ctlq_shutdown(hw, cq);
+ idpf_free(hw, cq);
+}
+
+/**
+ * idpf_ctlq_init - main initialization routine for all control queues
+ * @hw: pointer to hardware struct
+ * @num_q: number of queues to initialize
+ * @q_info: array of structs containing info for each queue to be initialized
+ *
+ * This initializes any number and any type of control queues. This is an all
+ * or nothing routine; if one fails, all previously allocated queues will be
+ * destroyed. This must be called prior to using the individual add/remove
+ * APIs.
+ */
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+ int i = 0;
+
+ LIST_INIT(&hw->cq_list_head);
+
+ for (i = 0; i < num_q; i++) {
+ struct idpf_ctlq_create_info *qinfo = q_info + i;
+
+ ret_code = idpf_ctlq_add(hw, qinfo, &cq);
+ if (ret_code)
+ goto init_destroy_qs;
+ }
+
+ return ret_code;
+
+init_destroy_qs:
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_deinit - destroy all control queues
+ * @hw: pointer to hw struct
+ */
+int idpf_ctlq_deinit(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_info *cq = NULL, *tmp = NULL;
+ int ret_code = 0;
+
+ LIST_FOR_EACH_ENTRY_SAFE(cq, tmp, &hw->cq_list_head,
+ idpf_ctlq_info, cq_list)
+ idpf_ctlq_remove(hw, cq);
+
+ return ret_code;
+}
+
+/**
+ * idpf_ctlq_send - send command to Control Queue (CTQ)
+ * @hw: pointer to hw struct
+ * @cq: handle to control queue struct to send on
+ * @num_q_msg: number of messages to send on control queue
+ * @q_msg: pointer to array of queue messages to be sent
+ *
+ * The caller is expected to allocate DMAable buffers and pass them to the
+ * send routine via the q_msg struct / control queue specific data struct.
+ * The control queue will hold a reference to each send message until
+ * the completion for that message has been cleaned.
+ */
+int idpf_ctlq_send(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 num_q_msg, struct idpf_ctlq_msg q_msg[])
+{
+ struct idpf_ctlq_desc *desc;
+ int num_desc_avail = 0;
+ int status = 0;
+ int i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ /* Ensure there are enough descriptors to send all messages */
+ num_desc_avail = IDPF_CTLQ_DESC_UNUSED(cq);
+ if (num_desc_avail == 0 || num_desc_avail < num_q_msg) {
+ status = -ENOSPC;
+ goto sq_send_command_out;
+ }
+
+ for (i = 0; i < num_q_msg; i++) {
+ struct idpf_ctlq_msg *msg = &q_msg[i];
+ u64 msg_cookie;
+
+ desc = IDPF_CTLQ_DESC(cq, cq->next_to_use);
+
+ desc->opcode = CPU_TO_LE16(msg->opcode);
+ desc->pfid_vfid = CPU_TO_LE16(msg->func_id);
+
+ msg_cookie = *(u64 *)&msg->cookie;
+ desc->cookie_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(msg_cookie));
+ desc->cookie_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(msg_cookie));
+
+ desc->flags = CPU_TO_LE16((msg->host_id & IDPF_HOST_ID_MASK) <<
+ IDPF_CTLQ_FLAG_HOST_ID_S);
+ if (msg->data_len) {
+ struct idpf_dma_mem *buff = msg->ctx.indirect.payload;
+
+ desc->datalen |= CPU_TO_LE16(msg->data_len);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF);
+ desc->flags |= CPU_TO_LE16(IDPF_CTLQ_FLAG_RD);
+
+ /* Update the address values in the desc with the pa
+ * value for respective buffer
+ */
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(buff->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(buff->pa));
+
+ idpf_memcpy(&desc->params, msg->ctx.indirect.context,
+ IDPF_INDIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.indirect.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ } else {
+ idpf_memcpy(&desc->params, msg->ctx.direct,
+ IDPF_DIRECT_CTX_SIZE, IDPF_NONDMA_TO_DMA);
+#ifdef SIMICS_BUILD
+ /* MBX message with opcode idpf_mbq_opc_send_msg_to_pf
+ * need to set peer PF function id in param0 for Simics
+ */
+ if (msg->opcode == idpf_mbq_opc_send_msg_to_pf) {
+ desc->params.direct.param0 =
+ CPU_TO_LE32(msg->func_id);
+ }
+#endif
+ }
+
+ /* Store buffer info */
+ cq->bi.tx_msg[cq->next_to_use] = msg;
+
+ (cq->next_to_use)++;
+ if (cq->next_to_use == cq->ring_size)
+ cq->next_to_use = 0;
+ }
+
+ /* Force memory write to complete before letting hardware
+ * know that there are new descriptors to fetch.
+ */
+ idpf_wmb();
+
+ wr32(hw, cq->reg.tail, cq->next_to_use);
+
+sq_send_command_out:
+ idpf_release_lock(&cq->cq_lock);
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_clean_sq - reclaim send descriptors on HW write back for the
+ * requested queue
+ * @cq: pointer to the specific Control queue
+ * @clean_count: (input|output) number of descriptors to clean as input, and
+ * number of descriptors actually cleaned as output
+ * @msg_status: (output) pointer to msg pointer array to be populated; needs
+ * to be allocated by caller
+ *
+ * Returns an array of message pointers associated with the cleaned
+ * descriptors. The pointers are to the original ctlq_msgs sent on the cleaned
+ * descriptors. The status will be returned for each; any messages that failed
+ * to send will have a non-zero status. The caller is expected to free original
+ * ctlq_msgs and free or reuse the DMA buffers.
+ */
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[])
+{
+ struct idpf_ctlq_desc *desc;
+ u16 i = 0, num_to_clean;
+ u16 ntc, desc_err;
+ int ret = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*clean_count == 0)
+ return 0;
+ if (*clean_count > cq->ring_size)
+ return -EINVAL;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *clean_count;
+
+ for (i = 0; i < num_to_clean; i++) {
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ if (!(LE16_TO_CPU(desc->flags) & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ desc_err = LE16_TO_CPU(desc->ret_val);
+ if (desc_err) {
+ /* strip off FW internal code */
+ desc_err &= 0xff;
+ }
+
+ msg_status[i] = cq->bi.tx_msg[ntc];
+ msg_status[i]->status = desc_err;
+
+ cq->bi.tx_msg[ntc] = NULL;
+
+ /* Zero out any stale data */
+ idpf_memset(desc, 0, sizeof(*desc), IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ }
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* Return number of descriptors actually cleaned */
+ *clean_count = i;
+
+ return ret;
+}
+
+/**
+ * idpf_ctlq_post_rx_buffs - post buffers to descriptor ring
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue handle
+ * @buff_count: (input|output) input is number of buffers caller is trying to
+ * return; output is number of buffers that were not posted
+ * @buffs: array of pointers to dma mem structs to be given to hardware
+ *
+ * Caller uses this function to return DMA buffers to the descriptor ring after
+ * consuming them; buff_count will be the number of buffers.
+ *
+ * Note: this function needs to be called after a receive call even
+ * if there are no DMA buffers to be returned, i.e. buff_count = 0,
+ * buffs = NULL to support direct commands
+ */
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw, struct idpf_ctlq_info *cq,
+ u16 *buff_count, struct idpf_dma_mem **buffs)
+{
+ struct idpf_ctlq_desc *desc;
+ u16 ntp = cq->next_to_post;
+ bool buffs_avail = false;
+ u16 tbp = ntp + 1;
+ int status = 0;
+ int i = 0;
+
+ if (*buff_count > cq->ring_size)
+ return -EINVAL;
+
+ if (*buff_count > 0)
+ buffs_avail = true;
+
+ idpf_acquire_lock(&cq->cq_lock);
+
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ if (tbp == cq->next_to_clean)
+ /* Nothing to do */
+ goto post_buffs_out;
+
+ /* Post buffers for as many as provided or up until the last one used */
+ while (ntp != cq->next_to_clean) {
+ desc = IDPF_CTLQ_DESC(cq, ntp);
+
+ if (cq->bi.rx_buff[ntp])
+ goto fill_desc;
+ if (!buffs_avail) {
+ /* If the caller hasn't given us any buffers or
+ * there are none left, search the ring itself
+ * for an available buffer to move to this
+ * entry starting at the next entry in the ring
+ */
+ tbp = ntp + 1;
+
+ /* Wrap ring if necessary */
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+
+ while (tbp != cq->next_to_clean) {
+ if (cq->bi.rx_buff[tbp]) {
+ cq->bi.rx_buff[ntp] =
+ cq->bi.rx_buff[tbp];
+ cq->bi.rx_buff[tbp] = NULL;
+
+ /* Found a buffer, no need to
+ * search anymore
+ */
+ break;
+ }
+
+ /* Wrap ring if necessary */
+ tbp++;
+ if (tbp >= cq->ring_size)
+ tbp = 0;
+ }
+
+ if (tbp == cq->next_to_clean)
+ goto post_buffs_out;
+ } else {
+ /* Give back pointer to DMA buffer */
+ cq->bi.rx_buff[ntp] = buffs[i];
+ i++;
+
+ if (i >= *buff_count)
+ buffs_avail = false;
+ }
+
+fill_desc:
+ desc->flags =
+ CPU_TO_LE16(IDPF_CTLQ_FLAG_BUF | IDPF_CTLQ_FLAG_RD);
+
+ /* Post buffers to descriptor */
+ desc->datalen = CPU_TO_LE16(cq->bi.rx_buff[ntp]->size);
+ desc->params.indirect.addr_high =
+ CPU_TO_LE32(IDPF_HI_DWORD(cq->bi.rx_buff[ntp]->pa));
+ desc->params.indirect.addr_low =
+ CPU_TO_LE32(IDPF_LO_DWORD(cq->bi.rx_buff[ntp]->pa));
+
+ ntp++;
+ if (ntp == cq->ring_size)
+ ntp = 0;
+ }
+
+post_buffs_out:
+ /* Only update tail if buffers were actually posted */
+ if (cq->next_to_post != ntp) {
+ if (ntp)
+ /* Update next_to_post to ntp - 1 since current ntp
+ * will not have a buffer
+ */
+ cq->next_to_post = ntp - 1;
+ else
+ /* Wrap to end of end ring since current ntp is 0 */
+ cq->next_to_post = cq->ring_size - 1;
+
+ wr32(hw, cq->reg.tail, cq->next_to_post);
+ }
+
+ idpf_release_lock(&cq->cq_lock);
+
+ /* return the number of buffers that were not posted */
+ *buff_count = *buff_count - i;
+
+ return status;
+}
+
+/**
+ * idpf_ctlq_recv - receive control queue message call back
+ * @cq: pointer to control queue handle to receive on
+ * @num_q_msg: (input|output) input number of messages that should be received;
+ * output number of messages actually received
+ * @q_msg: (output) array of received control queue messages on this q;
+ * needs to be pre-allocated by caller for as many messages as requested
+ *
+ * Called by interrupt handler or polling mechanism. Caller is expected
+ * to free buffers
+ */
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg)
+{
+ u16 num_to_clean, ntc, ret_val, flags;
+ struct idpf_ctlq_desc *desc;
+ int ret_code = 0;
+ u16 i = 0;
+
+ if (!cq || !cq->ring_size)
+ return -ENOBUFS;
+
+ if (*num_q_msg == 0)
+ return 0;
+ else if (*num_q_msg > cq->ring_size)
+ return -EINVAL;
+
+ /* take the lock before we start messing with the ring */
+ idpf_acquire_lock(&cq->cq_lock);
+
+ ntc = cq->next_to_clean;
+
+ num_to_clean = *num_q_msg;
+
+ for (i = 0; i < num_to_clean; i++) {
+ u64 msg_cookie;
+
+ /* Fetch next descriptor and check if marked as done */
+ desc = IDPF_CTLQ_DESC(cq, ntc);
+ flags = LE16_TO_CPU(desc->flags);
+
+ if (!(flags & IDPF_CTLQ_FLAG_DD))
+ break;
+
+ ret_val = LE16_TO_CPU(desc->ret_val);
+
+ q_msg[i].vmvf_type = (flags &
+ (IDPF_CTLQ_FLAG_FTYPE_VM |
+ IDPF_CTLQ_FLAG_FTYPE_PF)) >>
+ IDPF_CTLQ_FLAG_FTYPE_S;
+
+ if (flags & IDPF_CTLQ_FLAG_ERR)
+ ret_code = -EBADMSG;
+
+ msg_cookie = (u64)LE32_TO_CPU(desc->cookie_high) << 32;
+ msg_cookie |= (u64)LE32_TO_CPU(desc->cookie_low);
+ idpf_memcpy(&q_msg[i].cookie, &msg_cookie, sizeof(u64),
+ IDPF_NONDMA_TO_NONDMA);
+
+ q_msg[i].opcode = LE16_TO_CPU(desc->opcode);
+ q_msg[i].data_len = LE16_TO_CPU(desc->datalen);
+ q_msg[i].status = ret_val;
+
+ if (desc->datalen) {
+ idpf_memcpy(q_msg[i].ctx.indirect.context,
+ &desc->params.indirect,
+ IDPF_INDIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+
+ /* Assign pointer to dma buffer to ctlq_msg array
+ * to be given to upper layer
+ */
+ q_msg[i].ctx.indirect.payload = cq->bi.rx_buff[ntc];
+
+ /* Zero out pointer to DMA buffer info;
+ * will be repopulated by post buffers API
+ */
+ cq->bi.rx_buff[ntc] = NULL;
+ } else {
+ idpf_memcpy(q_msg[i].ctx.direct,
+ desc->params.raw,
+ IDPF_DIRECT_CTX_SIZE,
+ IDPF_DMA_TO_NONDMA);
+ }
+
+ /* Zero out stale data in descriptor */
+ idpf_memset(desc, 0, sizeof(struct idpf_ctlq_desc),
+ IDPF_DMA_MEM);
+
+ ntc++;
+ if (ntc == cq->ring_size)
+ ntc = 0;
+ };
+
+ cq->next_to_clean = ntc;
+
+ idpf_release_lock(&cq->cq_lock);
+
+ *num_q_msg = i;
+ if (*num_q_msg == 0)
+ ret_code = -ENOMSG;
+
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_controlq.h b/drivers/common/idpf/base/idpf_controlq.h
new file mode 100644
index 0000000000..e7b0d803b3
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_H_
+#define _IDPF_CONTROLQ_H_
+
+#ifdef __KERNEL__
+#include <linux/slab.h>
+#endif
+
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#include "idpf_alloc.h"
+#endif
+#include "idpf_controlq_api.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define IDPF_CTLQ_MAX_RING_SIZE 1024
+#define IDPF_CTLQ_MAX_BUF_LEN 4096
+
+#define IDPF_CTLQ_DESC(R, i) \
+ (&(((struct idpf_ctlq_desc *)((R)->desc_ring.va))[i]))
+
+#define IDPF_CTLQ_DESC_UNUSED(R) \
+ ((u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->ring_size) + \
+ (R)->next_to_clean - (R)->next_to_use - 1))
+
+#ifndef __KERNEL__
+/* Data type manipulation macros. */
+#define IDPF_HI_DWORD(x) ((u32)((((x) >> 16) >> 16) & 0xFFFFFFFF))
+#define IDPF_LO_DWORD(x) ((u32)((x) & 0xFFFFFFFF))
+#define IDPF_HI_WORD(x) ((u16)(((x) >> 16) & 0xFFFF))
+#define IDPF_LO_WORD(x) ((u16)((x) & 0xFFFF))
+
+#endif
+/* Control Queue default settings */
+#define IDPF_CTRL_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct idpf_ctlq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen; /* 0 for direct commands */
+ union {
+ __le16 ret_val;
+ __le16 pfid_vfid;
+#define IDPF_CTLQ_DESC_VF_ID_S 0
+#ifdef SIMICS_BUILD
+#define IDPF_CTLQ_DESC_VF_ID_M (0x3FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 10
+#define IDPF_CTLQ_DESC_PF_ID_M (0x3F << IDPF_CTLQ_DESC_PF_ID_S)
+#else
+#define IDPF_CTLQ_DESC_VF_ID_M (0x7FF << IDPF_CTLQ_DESC_VF_ID_S)
+#define IDPF_CTLQ_DESC_PF_ID_S 11
+#define IDPF_CTLQ_DESC_PF_ID_M (0x1F << IDPF_CTLQ_DESC_PF_ID_S)
+#endif
+ };
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 param2;
+ __le32 param3;
+ } direct;
+ struct {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+ } indirect;
+ u8 raw[16];
+ } params;
+};
+
+/* Flags sub-structure
+ * |0 |1 |2 |3 |4 |5 |6 |7 |8 |9 |10 |11 |12 |13 |14 |15 |
+ * |DD |CMP|ERR| * RSV * |FTYPE | *RSV* |RD |VFC|BUF| HOST_ID |
+ */
+/* command flags and offsets */
+#define IDPF_CTLQ_FLAG_DD_S 0
+#define IDPF_CTLQ_FLAG_CMP_S 1
+#define IDPF_CTLQ_FLAG_ERR_S 2
+#define IDPF_CTLQ_FLAG_FTYPE_S 6
+#define IDPF_CTLQ_FLAG_RD_S 10
+#define IDPF_CTLQ_FLAG_VFC_S 11
+#define IDPF_CTLQ_FLAG_BUF_S 12
+#define IDPF_CTLQ_FLAG_HOST_ID_S 13
+
+#define IDPF_CTLQ_FLAG_DD BIT(IDPF_CTLQ_FLAG_DD_S) /* 0x1 */
+#define IDPF_CTLQ_FLAG_CMP BIT(IDPF_CTLQ_FLAG_CMP_S) /* 0x2 */
+#define IDPF_CTLQ_FLAG_ERR BIT(IDPF_CTLQ_FLAG_ERR_S) /* 0x4 */
+#define IDPF_CTLQ_FLAG_FTYPE_VM BIT(IDPF_CTLQ_FLAG_FTYPE_S) /* 0x40 */
+#define IDPF_CTLQ_FLAG_FTYPE_PF BIT(IDPF_CTLQ_FLAG_FTYPE_S + 1) /* 0x80 */
+#define IDPF_CTLQ_FLAG_RD BIT(IDPF_CTLQ_FLAG_RD_S) /* 0x400 */
+#define IDPF_CTLQ_FLAG_VFC BIT(IDPF_CTLQ_FLAG_VFC_S) /* 0x800 */
+#define IDPF_CTLQ_FLAG_BUF BIT(IDPF_CTLQ_FLAG_BUF_S) /* 0x1000 */
+
+/* Host ID is a special field that has 3b and not a 1b flag */
+#define IDPF_CTLQ_FLAG_HOST_ID_M MAKE_MASK(0x7000UL, IDPF_CTLQ_FLAG_HOST_ID_S)
+
+struct idpf_mbxq_desc {
+ u8 pad[8]; /* CTLQ flags/opcode/len/retval fields */
+ u32 chnl_opcode; /* avoid confusion with desc->opcode */
+ u32 chnl_retval; /* ditto for desc->retval */
+ u32 pf_vf_id; /* used by CP when sending to PF */
+};
+
+enum idpf_mac_type {
+ IDPF_MAC_UNKNOWN = 0,
+ IDPF_MAC_PF,
+ IDPF_MAC_VF,
+ IDPF_MAC_GENERIC
+};
+
+#define ETH_ALEN 6
+
+struct idpf_mac_info {
+ enum idpf_mac_type type;
+ u8 addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+#define IDPF_AQ_LINK_UP 0x1
+
+/* PCI bus types */
+enum idpf_bus_type {
+ idpf_bus_type_unknown = 0,
+ idpf_bus_type_pci,
+ idpf_bus_type_pcix,
+ idpf_bus_type_pci_express,
+ idpf_bus_type_reserved
+};
+
+/* PCI bus speeds */
+enum idpf_bus_speed {
+ idpf_bus_speed_unknown = 0,
+ idpf_bus_speed_33 = 33,
+ idpf_bus_speed_66 = 66,
+ idpf_bus_speed_100 = 100,
+ idpf_bus_speed_120 = 120,
+ idpf_bus_speed_133 = 133,
+ idpf_bus_speed_2500 = 2500,
+ idpf_bus_speed_5000 = 5000,
+ idpf_bus_speed_8000 = 8000,
+ idpf_bus_speed_reserved
+};
+
+/* PCI bus widths */
+enum idpf_bus_width {
+ idpf_bus_width_unknown = 0,
+ idpf_bus_width_pcie_x1 = 1,
+ idpf_bus_width_pcie_x2 = 2,
+ idpf_bus_width_pcie_x4 = 4,
+ idpf_bus_width_pcie_x8 = 8,
+ idpf_bus_width_32 = 32,
+ idpf_bus_width_64 = 64,
+ idpf_bus_width_reserved
+};
+
+/* Bus parameters */
+struct idpf_bus_info {
+ enum idpf_bus_speed speed;
+ enum idpf_bus_width width;
+ enum idpf_bus_type type;
+
+ u16 func;
+ u16 device;
+ u16 lan_id;
+ u16 bus_id;
+};
+
+/* Function specific capabilities */
+struct idpf_hw_func_caps {
+ u32 num_alloc_vfs;
+ u32 vf_base_id;
+};
+
+/* Define the APF hardware struct to replace other control structs as needed
+ * Align to ctlq_hw_info
+ */
+struct idpf_hw {
+ /* Some part of BAR0 address space is not mapped by the LAN driver.
+ * This results in 2 regions of BAR0 to be mapped by LAN driver which
+ * will have its own base hardware address when mapped.
+ */
+ u8 *hw_addr;
+ u8 *hw_addr_region2;
+ u64 hw_addr_len;
+ u64 hw_addr_region2_len;
+
+ void *back;
+
+ /* control queue - send and receive */
+ struct idpf_ctlq_info *asq;
+ struct idpf_ctlq_info *arq;
+
+ /* subsystem structs */
+ struct idpf_mac_info mac;
+ struct idpf_bus_info bus;
+ struct idpf_hw_func_caps func_caps;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+ bool adapter_stopped;
+
+ LIST_HEAD_TYPE(list_head, idpf_ctlq_info) cq_list_head;
+};
+
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq);
+
+/* prototype for functions used for dynamic memory allocation */
+void *idpf_alloc_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem,
+ u64 size);
+void idpf_free_dma_mem(struct idpf_hw *hw, struct idpf_dma_mem *mem);
+#endif /* _IDPF_CONTROLQ_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_api.h b/drivers/common/idpf/base/idpf_controlq_api.h
new file mode 100644
index 0000000000..2d79c7f4dd
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_api.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_CONTROLQ_API_H_
+#define _IDPF_CONTROLQ_API_H_
+
+#ifdef __KERNEL__
+#include "idpf_mem.h"
+#else /* !__KERNEL__ */
+#include "idpf_osdep.h"
+#endif /* !__KERNEL__ */
+
+struct idpf_hw;
+
+/* Used for queue init, response and events */
+enum idpf_ctlq_type {
+ IDPF_CTLQ_TYPE_MAILBOX_TX = 0,
+ IDPF_CTLQ_TYPE_MAILBOX_RX = 1,
+ IDPF_CTLQ_TYPE_CONFIG_TX = 2,
+ IDPF_CTLQ_TYPE_CONFIG_RX = 3,
+ IDPF_CTLQ_TYPE_EVENT_RX = 4,
+ IDPF_CTLQ_TYPE_RDMA_TX = 5,
+ IDPF_CTLQ_TYPE_RDMA_RX = 6,
+ IDPF_CTLQ_TYPE_RDMA_COMPL = 7
+};
+
+/*
+ * Generic Control Queue Structures
+ */
+
+struct idpf_ctlq_reg {
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ /* Below applies only to default mb (if present) */
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* Generic queue msg structure */
+struct idpf_ctlq_msg {
+ u8 vmvf_type; /* represents the source of the message on recv */
+#define IDPF_VMVF_TYPE_VF 0
+#define IDPF_VMVF_TYPE_VM 1
+#define IDPF_VMVF_TYPE_PF 2
+ u8 host_id;
+ /* 3b field used only when sending a message to peer - to be used in
+ * combination with target func_id to route the message
+ */
+#define IDPF_HOST_ID_MASK 0x7
+
+ u16 opcode;
+ u16 data_len; /* data_len = 0 when no payload is attached */
+ union {
+ u16 func_id; /* when sending a message */
+ u16 status; /* when receiving a message */
+ };
+ union {
+ struct {
+ u32 chnl_retval;
+ u32 chnl_opcode;
+ } mbx;
+ } cookie;
+ union {
+#define IDPF_DIRECT_CTX_SIZE 16
+#define IDPF_INDIRECT_CTX_SIZE 8
+ /* 16 bytes of context can be provided or 8 bytes of context
+ * plus the address of a DMA buffer
+ */
+ u8 direct[IDPF_DIRECT_CTX_SIZE];
+ struct {
+ u8 context[IDPF_INDIRECT_CTX_SIZE];
+ struct idpf_dma_mem *payload;
+ } indirect;
+ } ctx;
+};
+
+/* Generic queue info structures */
+/* MB, CONFIG and EVENT q do not have extended info */
+struct idpf_ctlq_create_info {
+ enum idpf_ctlq_type type;
+ int id; /* absolute queue offset passed as input
+ * -1 for default mailbox if present
+ */
+ u16 len; /* Queue length passed as input */
+ u16 buf_size; /* buffer size passed as input */
+ u64 base_address; /* output, HPA of the Queue start */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+
+ int ext_info_size;
+ void *ext_info; /* Specific to q type */
+};
+
+/* Control Queue information */
+struct idpf_ctlq_info {
+ LIST_ENTRY_TYPE(idpf_ctlq_info) cq_list;
+
+ enum idpf_ctlq_type cq_type;
+ int q_id;
+ idpf_lock cq_lock; /* queue lock
+ * idpf_lock is defined in OSdep.h
+ */
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+ u16 next_to_post; /* starting descriptor to post buffers
+ * to after recev
+ */
+
+ struct idpf_dma_mem desc_ring; /* descriptor ring memory
+ * idpf_dma_mem is defined in OSdep.h
+ */
+ union {
+ struct idpf_dma_mem **rx_buff;
+ struct idpf_ctlq_msg **tx_msg;
+ } bi;
+
+ u16 buf_size; /* queue buffer size */
+ u16 ring_size; /* Number of descriptors */
+ struct idpf_ctlq_reg reg; /* registers accessed by ctlqs */
+};
+
+/* PF/VF mailbox commands */
+enum idpf_mbx_opc {
+ /* idpf_mbq_opc_send_msg_to_pf:
+ * usage: used by PF or VF to send a message to its CPF
+ * target: RX queue and function ID of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_pf = 0x0801,
+
+ /* idpf_mbq_opc_send_msg_to_vf:
+ * usage: used by PF to send message to a VF
+ * target: VF control queue ID must be specified in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_vf = 0x0802,
+
+ /* idpf_mbq_opc_send_msg_to_peer_pf:
+ * usage: used by any function to send message to any peer PF
+ * target: RX queue and host of parent PF taken from HW
+ */
+ idpf_mbq_opc_send_msg_to_peer_pf = 0x0803,
+
+ /* idpf_mbq_opc_send_msg_to_peer_drv:
+ * usage: used by any function to send message to any peer driver
+ * target: RX queue and target host must be specific in descriptor
+ */
+ idpf_mbq_opc_send_msg_to_peer_drv = 0x0804,
+};
+
+/*
+ * API supported for control queue management
+ */
+
+/* Will init all required q including default mb. "q_info" is an array of
+ * create_info structs equal to the number of control queues to be created.
+ */
+__rte_internal
+int idpf_ctlq_init(struct idpf_hw *hw, u8 num_q,
+ struct idpf_ctlq_create_info *q_info);
+
+/* Allocate and initialize a single control queue, which will be added to the
+ * control queue list; returns a handle to the created control queue
+ */
+int idpf_ctlq_add(struct idpf_hw *hw,
+ struct idpf_ctlq_create_info *qinfo,
+ struct idpf_ctlq_info **cq);
+
+/* Deinitialize and deallocate a single control queue */
+void idpf_ctlq_remove(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq);
+
+/* Sends messages to HW and will also free the buffer*/
+__rte_internal
+int idpf_ctlq_send(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 num_q_msg,
+ struct idpf_ctlq_msg q_msg[]);
+
+/* Receives messages and called by interrupt handler/polling
+ * initiated by app/process. Also caller is supposed to free the buffers
+ */
+__rte_internal
+int idpf_ctlq_recv(struct idpf_ctlq_info *cq, u16 *num_q_msg,
+ struct idpf_ctlq_msg *q_msg);
+
+/* Reclaims send descriptors on HW write back */
+__rte_internal
+int idpf_ctlq_clean_sq(struct idpf_ctlq_info *cq, u16 *clean_count,
+ struct idpf_ctlq_msg *msg_status[]);
+
+/* Indicate RX buffers are done being processed */
+__rte_internal
+int idpf_ctlq_post_rx_buffs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq,
+ u16 *buff_count,
+ struct idpf_dma_mem **buffs);
+
+/* Will destroy all q including the default mb */
+__rte_internal
+int idpf_ctlq_deinit(struct idpf_hw *hw);
+
+#endif /* _IDPF_CONTROLQ_API_H_ */
diff --git a/drivers/common/idpf/base/idpf_controlq_setup.c b/drivers/common/idpf/base/idpf_controlq_setup.c
new file mode 100644
index 0000000000..3a272b1f8d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_controlq_setup.c
@@ -0,0 +1,179 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+
+#include "idpf_controlq.h"
+
+
+/**
+ * idpf_ctlq_alloc_desc_ring - Allocate Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ */
+static int
+idpf_ctlq_alloc_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ size_t size = cq->ring_size * sizeof(struct idpf_ctlq_desc);
+
+ cq->desc_ring.va = idpf_alloc_dma_mem(hw, &cq->desc_ring, size);
+ if (!cq->desc_ring.va)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * idpf_ctlq_alloc_bufs - Allocate Control Queue (CQ) buffers
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Allocate the buffer head for all control queues, and if it's a receive
+ * queue, allocate DMA buffers
+ */
+static int idpf_ctlq_alloc_bufs(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ int i = 0;
+
+ /* Do not allocate DMA buffers for transmit queues */
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ return 0;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->bi.rx_buff = (struct idpf_dma_mem **)
+ idpf_calloc(hw, cq->ring_size,
+ sizeof(struct idpf_dma_mem *));
+ if (!cq->bi.rx_buff)
+ return -ENOMEM;
+
+ /* allocate the mapped buffers (except for the last one) */
+ for (i = 0; i < cq->ring_size - 1; i++) {
+ struct idpf_dma_mem *bi;
+ int num = 1; /* number of idpf_dma_mem to be allocated */
+
+ cq->bi.rx_buff[i] = (struct idpf_dma_mem *)idpf_calloc(hw, num,
+ sizeof(struct idpf_dma_mem));
+ if (!cq->bi.rx_buff[i])
+ goto unwind_alloc_cq_bufs;
+
+ bi = cq->bi.rx_buff[i];
+
+ bi->va = idpf_alloc_dma_mem(hw, bi, cq->buf_size);
+ if (!bi->va) {
+ /* unwind will not free the failed entry */
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ goto unwind_alloc_cq_bufs;
+ }
+ }
+
+ return 0;
+
+unwind_alloc_cq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ idpf_free(hw, cq->bi.rx_buff);
+
+ return -ENOMEM;
+}
+
+/**
+ * idpf_ctlq_free_desc_ring - Free Control Queue (CQ) rings
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void idpf_ctlq_free_desc_ring(struct idpf_hw *hw,
+ struct idpf_ctlq_info *cq)
+{
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+}
+
+/**
+ * idpf_ctlq_free_bufs - Free CQ buffer info elements
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the DMA buffers for RX queues, and DMA buffer header for both RX and TX
+ * queues. The upper layers are expected to manage freeing of TX DMA buffers
+ */
+static void idpf_ctlq_free_bufs(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ void *bi;
+
+ if (cq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX) {
+ int i;
+
+ /* free DMA buffers for rx queues*/
+ for (i = 0; i < cq->ring_size; i++) {
+ if (cq->bi.rx_buff[i]) {
+ idpf_free_dma_mem(hw, cq->bi.rx_buff[i]);
+ idpf_free(hw, cq->bi.rx_buff[i]);
+ }
+ }
+
+ bi = (void *)cq->bi.rx_buff;
+ } else {
+ bi = (void *)cq->bi.tx_msg;
+ }
+
+ /* free the buffer header */
+ idpf_free(hw, bi);
+}
+
+/**
+ * idpf_ctlq_dealloc_ring_res - Free memory allocated for control queue
+ * @hw: pointer to hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Free the memory used by the ring, buffers and other related structures
+ */
+void idpf_ctlq_dealloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ /* free ring buffers and the ring itself */
+ idpf_ctlq_free_bufs(hw, cq);
+ idpf_ctlq_free_desc_ring(hw, cq);
+}
+
+/**
+ * idpf_ctlq_alloc_ring_res - allocate memory for descriptor ring and bufs
+ * @hw: pointer to hw struct
+ * @cq: pointer to control queue struct
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+int idpf_ctlq_alloc_ring_res(struct idpf_hw *hw, struct idpf_ctlq_info *cq)
+{
+ int ret_code;
+
+ /* verify input for valid configuration */
+ if (!cq->ring_size || !cq->buf_size)
+ return -EINVAL;
+
+ /* allocate the ring memory */
+ ret_code = idpf_ctlq_alloc_desc_ring(hw, cq);
+ if (ret_code)
+ return ret_code;
+
+ /* allocate buffers in the rings */
+ ret_code = idpf_ctlq_alloc_bufs(hw, cq);
+ if (ret_code)
+ goto idpf_init_cq_free_ring;
+
+ /* success! */
+ return 0;
+
+idpf_init_cq_free_ring:
+ idpf_free_dma_mem(hw, &cq->desc_ring);
+ return ret_code;
+}
diff --git a/drivers/common/idpf/base/idpf_devids.h b/drivers/common/idpf/base/idpf_devids.h
new file mode 100644
index 0000000000..a91eb4e02a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_devids.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_DEVIDS_H_
+#define _IDPF_DEVIDS_H_
+
+/* Vendor ID */
+#define IDPF_INTEL_VENDOR_ID 0x8086
+
+/* Device IDs */
+#define IDPF_DEV_ID_PF 0x1452
+#define IDPF_DEV_ID_VF 0x1889
+
+
+
+
+#endif /* _IDPF_DEVIDS_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_pf_regs.h b/drivers/common/idpf/base/idpf_lan_pf_regs.h
new file mode 100644
index 0000000000..3df2347bd7
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_pf_regs.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_PF_REGS_H_
+#define _IDPF_LAN_PF_REGS_H_
+
+
+/* Receive queues */
+#define PF_QRX_BASE 0x00000000
+#define PF_QRX_TAIL(_QRX) (PF_QRX_BASE + (((_QRX) * 0x1000)))
+#define PF_QRX_BUFFQ_BASE 0x03000000
+#define PF_QRX_BUFFQ_TAIL(_QRX) (PF_QRX_BUFFQ_BASE + (((_QRX) * 0x1000)))
+
+/* Transmit queues */
+#define PF_QTX_BASE 0x05000000
+#define PF_QTX_COMM_DBELL(_DBQM) (PF_QTX_BASE + ((_DBQM) * 0x1000))
+
+
+/* Control(PF Mailbox) Queue */
+#define PF_FW_BASE 0x08400000
+
+#define PF_FW_ARQBAL (PF_FW_BASE)
+#define PF_FW_ARQBAH (PF_FW_BASE + 0x4)
+#define PF_FW_ARQLEN (PF_FW_BASE + 0x8)
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M MAKEMASK(0x1FFF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQH (PF_FW_BASE + 0xC)
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M MAKEMASK(0x1FFF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQT (PF_FW_BASE + 0x10)
+
+#define PF_FW_ATQBAL (PF_FW_BASE + 0x14)
+#define PF_FW_ATQBAH (PF_FW_BASE + 0x18)
+#define PF_FW_ATQLEN (PF_FW_BASE + 0x1C)
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQH (PF_FW_BASE + 0x20)
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M MAKEMASK(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQT (PF_FW_BASE + 0x24)
+
+/* Interrupts */
+#define PF_GLINT_BASE 0x08900000
+#define PF_GLINT_DYN_CTL(_INT) (PF_GLINT_BASE + ((_INT) * 0x1000))
+#define PF_GLINT_DYN_CTL_INTENA_S 0
+#define PF_GLINT_DYN_CTL_INTENA_M BIT(PF_GLINT_DYN_CTL_INTENA_S)
+#define PF_GLINT_DYN_CTL_CLEARPBA_S 1
+#define PF_GLINT_DYN_CTL_CLEARPBA_M BIT(PF_GLINT_DYN_CTL_CLEARPBA_S)
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define PF_GLINT_DYN_CTL_SWINT_TRIG_M BIT(PF_GLINT_DYN_CTL_SWINT_TRIG_S)
+#define PF_GLINT_DYN_CTL_ITR_INDX_S 3
+#define PF_GLINT_DYN_CTL_ITR_INDX_M MAKEMASK(0x3, PF_GLINT_DYN_CTL_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_INTERVAL_S 5
+#define PF_GLINT_DYN_CTL_INTERVAL_M BIT(PF_GLINT_DYN_CTL_INTERVAL_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S 24
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_ENA_S)
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define PF_GLINT_DYN_CTL_SW_ITR_INDX_M BIT(PF_GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_S 30
+#define PF_GLINT_DYN_CTL_WB_ON_ITR_M BIT(PF_GLINT_DYN_CTL_WB_ON_ITR_S)
+#define PF_GLINT_DYN_CTL_INTENA_MSK_S 31
+#define PF_GLINT_DYN_CTL_INTENA_MSK_M BIT(PF_GLINT_DYN_CTL_INTENA_MSK_S)
+#define PF_GLINT_ITR_V2(_i, _reg_start) (((_i) * 4) + (_reg_start))
+#define PF_GLINT_ITR(_i, _INT) (PF_GLINT_BASE + (((_i) + 1) * 4) + ((_INT) * 0x1000))
+#define PF_GLINT_ITR_MAX_INDEX 2
+#define PF_GLINT_ITR_INTERVAL_S 0
+#define PF_GLINT_ITR_INTERVAL_M MAKEMASK(0xFFF, PF_GLINT_ITR_INTERVAL_S)
+
+/* Timesync registers */
+#define PF_TIMESYNC_BASE 0x08404000
+#define PF_GLTSYN_CMD_SYNC (PF_TIMESYNC_BASE)
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_S 0
+#define PF_GLTSYN_CMD_SYNC_EXEC_CMD_M MAKEMASK(0x3, PF_GLTSYN_CMD_SYNC_EXEC_CMD_S)
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_S 2
+#define PF_GLTSYN_CMD_SYNC_SHTIME_EN_M BIT(PF_GLTSYN_CMD_SYNC_SHTIME_EN_S)
+#define PF_GLTSYN_SHTIME_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H (PF_TIMESYNC_BASE + 0xC)
+#define PF_GLTSYN_ART_L (PF_TIMESYNC_BASE + 0x10)
+#define PF_GLTSYN_ART_H (PF_TIMESYNC_BASE + 0x14)
+
+/* Generic registers */
+#define PF_INT_DIR_OICR_ENA 0x08406000
+#define PF_INT_DIR_OICR_ENA_S 0
+#define PF_INT_DIR_OICR_ENA_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_ENA_S)
+#define PF_INT_DIR_OICR 0x08406004
+#define PF_INT_DIR_OICR_TSYN_EVNT 0
+#define PF_INT_DIR_OICR_PHY_TS_0 BIT(1)
+#define PF_INT_DIR_OICR_PHY_TS_1 BIT(2)
+#define PF_INT_DIR_OICR_CAUSE 0x08406008
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_S 0
+#define PF_INT_DIR_OICR_CAUSE_CAUSE_M MAKEMASK(0xFFFFFFFF, PF_INT_DIR_OICR_CAUSE_CAUSE_S)
+#define PF_INT_PBA_CLEAR 0x0840600C
+
+#define PF_FUNC_RID 0x08406010
+#define PF_FUNC_RID_FUNCTION_NUMBER_S 0
+#define PF_FUNC_RID_FUNCTION_NUMBER_M MAKEMASK(0x7, PF_FUNC_RID_FUNCTION_NUMBER_S)
+#define PF_FUNC_RID_DEVICE_NUMBER_S 3
+#define PF_FUNC_RID_DEVICE_NUMBER_M MAKEMASK(0x1F, PF_FUNC_RID_DEVICE_NUMBER_S)
+#define PF_FUNC_RID_BUS_NUMBER_S 8
+#define PF_FUNC_RID_BUS_NUMBER_M MAKEMASK(0xFF, PF_FUNC_RID_BUS_NUMBER_S)
+
+/* Reset registers */
+#define PFGEN_RTRIG 0x08407000
+#define PFGEN_RTRIG_CORER_S 0
+#define PFGEN_RTRIG_CORER_M BIT(0)
+#define PFGEN_RTRIG_LINKR_S 1
+#define PFGEN_RTRIG_LINKR_M BIT(1)
+#define PFGEN_RTRIG_IMCR_S 2
+#define PFGEN_RTRIG_IMCR_M BIT(2)
+#define PFGEN_RSTAT 0x08407008 /* PFR Status */
+#define PFGEN_RSTAT_PFR_STATE_S 0
+#define PFGEN_RSTAT_PFR_STATE_M MAKEMASK(0x3, PFGEN_RSTAT_PFR_STATE_S)
+#define PFGEN_CTRL 0x0840700C
+#define PFGEN_CTRL_PFSWR BIT(0)
+
+#endif
diff --git a/drivers/common/idpf/base/idpf_lan_txrx.h b/drivers/common/idpf/base/idpf_lan_txrx.h
new file mode 100644
index 0000000000..98484b267c
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_txrx.h
@@ -0,0 +1,428 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_TXRX_H_
+#define _IDPF_LAN_TXRX_H_
+#ifndef __KERNEL__
+#include "idpf_osdep.h"
+#endif
+
+enum idpf_rss_hash {
+ /* Values 0 - 28 are reserved for future use */
+ IDPF_HASH_INVALID = 0,
+ IDPF_HASH_NONF_UNICAST_IPV4_UDP = 29,
+ IDPF_HASH_NONF_MULTICAST_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_UDP,
+ IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV4_TCP,
+ IDPF_HASH_NONF_IPV4_SCTP,
+ IDPF_HASH_NONF_IPV4_OTHER,
+ IDPF_HASH_FRAG_IPV4,
+ /* Values 37-38 are reserved */
+ IDPF_HASH_NONF_UNICAST_IPV6_UDP = 39,
+ IDPF_HASH_NONF_MULTICAST_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_UDP,
+ IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK,
+ IDPF_HASH_NONF_IPV6_TCP,
+ IDPF_HASH_NONF_IPV6_SCTP,
+ IDPF_HASH_NONF_IPV6_OTHER,
+ IDPF_HASH_FRAG_IPV6,
+ IDPF_HASH_NONF_RSVD47,
+ IDPF_HASH_NONF_FCOE_OX,
+ IDPF_HASH_NONF_FCOE_RX,
+ IDPF_HASH_NONF_FCOE_OTHER,
+ /* Values 51-62 are reserved */
+ IDPF_HASH_L2_PAYLOAD = 63,
+ IDPF_HASH_MAX
+};
+
+/* Supported RSS offloads */
+#define IDPF_DEFAULT_RSS_HASH ( \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV4) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_SCTP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_OTHER) | \
+ BIT_ULL(IDPF_HASH_FRAG_IPV6) | \
+ BIT_ULL(IDPF_HASH_L2_PAYLOAD))
+
+ /* TODO: Wrap below comment under internal flag
+ * Below 6 pcktypes are not supported by FVL or older products
+ * They are supported by FPK and future products
+ */
+#define IDPF_DEFAULT_RSS_HASH_EXPANDED (IDPF_DEFAULT_RSS_HASH | \
+ BIT_ULL(IDPF_HASH_NONF_IPV4_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV4_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_IPV6_TCP_SYN_NO_ACK) | \
+ BIT_ULL(IDPF_HASH_NONF_UNICAST_IPV6_UDP) | \
+ BIT_ULL(IDPF_HASH_NONF_MULTICAST_IPV6_UDP))
+
+/* For idpf_splitq_base_tx_compl_desc */
+#define IDPF_TXD_COMPLQ_GEN_S 15
+#define IDPF_TXD_COMPLQ_GEN_M BIT_ULL(IDPF_TXD_COMPLQ_GEN_S)
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_S 11
+#define IDPF_TXD_COMPLQ_COMPL_TYPE_M \
+ MAKEMASK(0x7UL, IDPF_TXD_COMPLQ_COMPL_TYPE_S)
+#define IDPF_TXD_COMPLQ_QID_S 0
+#define IDPF_TXD_COMPLQ_QID_M MAKEMASK(0x3FFUL, IDPF_TXD_COMPLQ_QID_S)
+
+/* For base mode TX descriptors */
+
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S 23
+#define IDPF_TXD_CTX_QW0_TUNN_L4T_CS_M BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_L4T_CS_S)
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_S 19
+#define IDPF_TXD_CTX_QW0_TUNN_DECTTL_M \
+ (0xFULL << IDPF_TXD_CTX_QW0_TUNN_DECTTL_S)
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_S 12
+#define IDPF_TXD_CTX_QW0_TUNN_NATLEN_M \
+ (0X7FULL << IDPF_TXD_CTX_QW0_TUNN_NATLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S 11
+#define IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M \
+ BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_S)
+#define IDPF_TXD_CTX_EIP_NOINC_IPID_CONST \
+ IDPF_TXD_CTX_QW0_TUNN_EIP_NOINC_M
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_S 9
+#define IDPF_TXD_CTX_QW0_TUNN_NATT_M (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_UDP_TUNNELING BIT_ULL(IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_GRE_TUNNELING (0x2ULL << IDPF_TXD_CTX_QW0_TUNN_NATT_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S 2
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_M \
+ (0x3FULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IPLEN_S)
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S 0
+#define IDPF_TXD_CTX_QW0_TUNN_EXT_IP_M \
+ (0x3ULL << IDPF_TXD_CTX_QW0_TUNN_EXT_IP_S)
+
+#define IDPF_TXD_CTX_QW1_MSS_S 50
+#define IDPF_TXD_CTX_QW1_MSS_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_CTX_QW1_MSS_S)
+#define IDPF_TXD_CTX_QW1_TSO_LEN_S 30
+#define IDPF_TXD_CTX_QW1_TSO_LEN_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_CTX_QW1_TSO_LEN_S)
+#define IDPF_TXD_CTX_QW1_CMD_S 4
+#define IDPF_TXD_CTX_QW1_CMD_M \
+ MAKEMASK(0xFFFUL, IDPF_TXD_CTX_QW1_CMD_S)
+#define IDPF_TXD_CTX_QW1_DTYPE_S 0
+#define IDPF_TXD_CTX_QW1_DTYPE_M \
+ MAKEMASK(0xFUL, IDPF_TXD_CTX_QW1_DTYPE_S)
+#define IDPF_TXD_QW1_L2TAG1_S 48
+#define IDPF_TXD_QW1_L2TAG1_M \
+ MAKEMASK(0xFFFFULL, IDPF_TXD_QW1_L2TAG1_S)
+#define IDPF_TXD_QW1_TX_BUF_SZ_S 34
+#define IDPF_TXD_QW1_TX_BUF_SZ_M \
+ MAKEMASK(0x3FFFULL, IDPF_TXD_QW1_TX_BUF_SZ_S)
+#define IDPF_TXD_QW1_OFFSET_S 16
+#define IDPF_TXD_QW1_OFFSET_M \
+ MAKEMASK(0x3FFFFULL, IDPF_TXD_QW1_OFFSET_S)
+#define IDPF_TXD_QW1_CMD_S 4
+#define IDPF_TXD_QW1_CMD_M MAKEMASK(0xFFFUL, IDPF_TXD_QW1_CMD_S)
+#define IDPF_TXD_QW1_DTYPE_S 0
+#define IDPF_TXD_QW1_DTYPE_M MAKEMASK(0xFUL, IDPF_TXD_QW1_DTYPE_S)
+
+/* TX Completion Descriptor Completion Types */
+#define IDPF_TXD_COMPLT_ITR_FLUSH 0
+#define IDPF_TXD_COMPLT_RULE_MISS 1
+#define IDPF_TXD_COMPLT_RS 2
+#define IDPF_TXD_COMPLT_REINJECTED 3
+#define IDPF_TXD_COMPLT_RE 4
+#define IDPF_TXD_COMPLT_SW_MARKER 5
+
+enum idpf_tx_desc_dtype_value {
+ IDPF_TX_DESC_DTYPE_DATA = 0,
+ IDPF_TX_DESC_DTYPE_CTX = 1,
+ IDPF_TX_DESC_DTYPE_REINJECT_CTX = 2,
+ IDPF_TX_DESC_DTYPE_FLEX_DATA = 3,
+ IDPF_TX_DESC_DTYPE_FLEX_CTX = 4,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX = 5,
+ IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 = 6,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 = 7,
+ IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX = 8,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_TSO_CTX = 9,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_SA_CTX = 10,
+ IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX = 11,
+ IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE = 12,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_TSO_CTX = 13,
+ IDPF_TX_DESC_DTYPE_FLEX_HOSTSPLIT_CTX = 14,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ IDPF_TX_DESC_DTYPE_DESC_DONE = 15,
+};
+
+enum idpf_tx_ctx_desc_cmd_bits {
+ IDPF_TX_CTX_DESC_TSO = 0x01,
+ IDPF_TX_CTX_DESC_TSYN = 0x02,
+ IDPF_TX_CTX_DESC_IL2TAG2 = 0x04,
+ IDPF_TX_CTX_DESC_RSVD = 0x08,
+ IDPF_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ IDPF_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ IDPF_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ IDPF_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ IDPF_TX_CTX_DESC_FILT_AU_EN = 0x40,
+ IDPF_TX_CTX_DESC_FILT_AU_EVICT = 0x80,
+ IDPF_TX_CTX_DESC_RSVD1 = 0xF00
+};
+
+enum idpf_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ IDPF_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ IDPF_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ IDPF_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define IDPF_TXD_QW1_MACLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_MACLEN_S)
+#define IDPF_TXD_QW1_IPLEN_M MAKEMASK(0x7FUL, IDPF_TX_DESC_LEN_IPLEN_S)
+#define IDPF_TXD_QW1_L4LEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+#define IDPF_TXD_QW1_FCLEN_M MAKEMASK(0xFUL, IDPF_TX_DESC_LEN_L4_LEN_S)
+
+enum idpf_tx_base_desc_cmd_bits {
+ IDPF_TX_DESC_CMD_EOP = 0x0001,
+ IDPF_TX_DESC_CMD_RS = 0x0002,
+ /* only on VFs else RSVD */
+ IDPF_TX_DESC_CMD_ICRC = 0x0004,
+ IDPF_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ IDPF_TX_DESC_CMD_RSVD1 = 0x0010,
+ IDPF_TX_DESC_CMD_IIPT_NONIP = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ IDPF_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD2 = 0x0080,
+ IDPF_TX_DESC_CMD_L4T_EOFT_UNK = 0x0000, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_SCTP = 0x0200, /* 2 BITS */
+ IDPF_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+ IDPF_TX_DESC_CMD_RSVD3 = 0x0400,
+ IDPF_TX_DESC_CMD_RSVD4 = 0x0800,
+};
+
+/* Transmit descriptors */
+/* splitq tx buf, singleq tx buf and singleq compl desc */
+struct idpf_base_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 qw1; /* type_cmd_offset_bsz_l2tag1 */
+};/* read used with buffer queues*/
+
+struct idpf_splitq_tx_compl_desc {
+ /* qid=[10:0] comptype=[13:11] rsvd=[14] gen=[15] */
+ __le16 qid_comptype_gen;
+ union {
+ __le16 q_head; /* Queue head */
+ __le16 compl_tag; /* Completion tag */
+ } q_head_compl_tag;
+ u32 rsvd;
+
+};/* writeback used with completion queues*/
+
+/* Context descriptors */
+struct idpf_base_tx_ctx_desc {
+ struct {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd1;
+ } qw0;
+ __le64 qw1; /* type_cmd_tlen_mss/rt_hint */
+};
+
+/* Common cmd field defines for all desc except Flex Flow Scheduler (0x0C) */
+enum idpf_tx_flex_desc_cmd_bits {
+ IDPF_TX_FLEX_DESC_CMD_EOP = 0x01,
+ IDPF_TX_FLEX_DESC_CMD_RS = 0x02,
+ IDPF_TX_FLEX_DESC_CMD_RE = 0x04,
+ IDPF_TX_FLEX_DESC_CMD_IL2TAG1 = 0x08,
+ IDPF_TX_FLEX_DESC_CMD_DUMMY = 0x10,
+ IDPF_TX_FLEX_DESC_CMD_CS_EN = 0x20,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EN = 0x40,
+ IDPF_TX_FLEX_DESC_CMD_FILT_AU_EVICT = 0x80,
+};
+
+struct idpf_flex_tx_desc {
+ __le64 buf_addr; /* Packet buffer address */
+ struct {
+ __le16 cmd_dtype;
+#define IDPF_FLEX_TXD_QW1_DTYPE_S 0
+#define IDPF_FLEX_TXD_QW1_DTYPE_M \
+ MAKEMASK(0x1FUL, IDPF_FLEX_TXD_QW1_DTYPE_S)
+#define IDPF_FLEX_TXD_QW1_CMD_S 5
+#define IDPF_FLEX_TXD_QW1_CMD_M MAKEMASK(0x7FFUL, IDPF_TXD_QW1_CMD_S)
+ union {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_DATA_(0x03) */
+ u8 raw[4];
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSYN_L2TAG1 (0x06) */
+ struct {
+ __le16 l2tag1;
+ u8 flex;
+ u8 tsync;
+ } tsync;
+
+ /* DTYPE=IDPF_TX_DESC_DTYPE_FLEX_L2TAG1_L2TAG2 (0x07) */
+ struct {
+ __le16 l2tag1;
+ __le16 l2tag2;
+ } l2tags;
+ } flex;
+ __le16 buf_size;
+ } qw1;
+};
+
+struct idpf_flex_tx_sched_desc {
+ __le64 buf_addr; /* Packet buffer address */
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE_16B (0x0C) */
+ struct {
+ u8 cmd_dtype;
+#define IDPF_TXD_FLEX_FLOW_DTYPE_M 0x1F
+#define IDPF_TXD_FLEX_FLOW_CMD_EOP 0x20
+#define IDPF_TXD_FLEX_FLOW_CMD_CS_EN 0x40
+#define IDPF_TXD_FLEX_FLOW_CMD_RE 0x80
+
+ u8 rsvd[3];
+
+ __le16 compl_tag;
+ __le16 rxr_bufsize;
+#define IDPF_TXD_FLEX_FLOW_RXR 0x4000
+#define IDPF_TXD_FLEX_FLOW_BUFSIZE_M 0x3FFF
+ } qw1;
+};
+
+/* Common cmd fields for all flex context descriptors
+ * Note: these defines already account for the 5 bit dtype in the cmd_dtype
+ * field
+ */
+enum idpf_tx_flex_ctx_desc_cmd_bits {
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO = 0x0020,
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSYN_EN = 0x0040,
+ IDPF_TX_FLEX_CTX_DESC_CMD_L2TAG2 = 0x0080,
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_UPLNK = 0x0200, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_LOCAL = 0x0400, /* 2 bits */
+ IDPF_TX_FLEX_CTX_DESC_CMD_SWTCH_TARGETVSI = 0x0600, /* 2 bits */
+};
+
+/* Standard flex descriptor TSO context quad word */
+struct idpf_flex_tx_tso_ctx_qw {
+ __le32 flex_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_M 0x3FFFF
+#define IDPF_TXD_FLEX_TSO_CTX_FLEX_S 24
+ __le16 mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+ u8 hdr_len;
+ u8 flex;
+};
+
+union idpf_flex_tx_ctx_desc {
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_CTX (0x04) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag1;
+ u8 qw1_flex[4];
+ } qw1;
+ } gen;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX (0x05) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ u8 flex[6];
+ } qw1;
+ } tso;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_TSO_L2TAG2_PARSTAG_CTX (0x08) */
+ struct {
+ struct idpf_flex_tx_tso_ctx_qw qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex0;
+ u8 ptag;
+ u8 flex1[2];
+ } qw1;
+ } tso_l2tag2_ptag;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_FLEX_L2TAG2_CTX (0x0B) */
+ struct {
+ u8 qw0_flex[8];
+ struct {
+ __le16 cmd_dtype;
+ __le16 l2tag2;
+ u8 flex[4];
+ } qw1;
+ } l2tag2;
+
+ /* DTYPE = IDPF_TX_DESC_DTYPE_REINJECT_CTX (0x02) */
+ struct {
+ struct {
+ __le32 sa_domain;
+#define IDPF_TXD_FLEX_CTX_SA_DOM_M 0xFFFF
+#define IDPF_TXD_FLEX_CTX_SA_DOM_VAL 0x10000
+ __le32 sa_idx;
+#define IDPF_TXD_FLEX_CTX_SAIDX_M 0x1FFFFF
+ } qw0;
+ struct {
+ __le16 cmd_dtype;
+ __le16 txr2comp;
+#define IDPF_TXD_FLEX_CTX_TXR2COMP 0x1
+ __le16 miss_txq_comp_tag;
+ __le16 miss_txq_id;
+ } qw1;
+ } reinjection_pkt;
+};
+
+/* Host Split Context Descriptors */
+struct idpf_flex_tx_hs_ctx_desc {
+ union {
+ struct {
+ __le32 host_fnum_tlen;
+#define IDPF_TXD_FLEX_CTX_TLEN_S 0
+/* see IDPF_TXD_FLEX_CTX_TLEN_M for mask definition */
+#define IDPF_TXD_FLEX_CTX_FNUM_S 18
+#define IDPF_TXD_FLEX_CTX_FNUM_M 0x7FF
+#define IDPF_TXD_FLEX_CTX_HOST_S 29
+#define IDPF_TXD_FLEX_CTX_HOST_M 0x7
+ __le16 ftype_mss_rt;
+#define IDPF_TXD_FLEX_CTX_MSS_RT_0 0
+#define IDPF_TXD_FLEX_CTX_MSS_RT_M 0x3FFF
+#define IDPF_TXD_FLEX_CTX_FTYPE_S 14
+#define IDPF_TXD_FLEX_CTX_FTYPE_VF MAKEMASK(0x0, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_VDEV MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_FTYPE_S)
+#define IDPF_TXD_FLEX_CTX_FTYPE_PF MAKEMASK(0x2, IDPF_TXD_FLEX_CTX_FTYPE_S)
+ u8 hdr_len;
+ u8 ptag;
+ } tso;
+ struct {
+ u8 flex0[2];
+ __le16 host_fnum_ftype;
+ u8 flex1[3];
+ u8 ptag;
+ } no_tso;
+ } qw0;
+
+ __le64 qw1_cmd_dtype;
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_S 16
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_M 0xFFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID_S 36
+#define IDPF_TXD_FLEX_CTX_QW1_PASID_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_PASID_VALID_S)
+#define IDPF_TXD_FLEX_CTX_QW1_TPH_S 37
+#define IDPF_TXD_FLEX_CTX_QW1_TPH \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_TPH_S)
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_S 38
+#define IDPF_TXD_FLEX_CTX_QW1_PFNUM_M 0xF
+/* The following are only valid for DTYPE = 0x09 and DTYPE = 0x0A */
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_S 42
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_M 0x1FFFFF
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S 63
+#define IDPF_TXD_FLEX_CTX_QW1_SAIDX_VALID \
+ MAKEMASK(0x1, IDPF_TXD_FLEX_CTX_QW1_SAIDX_VAL_S)
+/* The following are only valid for DTYPE = 0x0D and DTYPE = 0x0E */
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_S 48
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX0_M 0xFF
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_S 56
+#define IDPF_TXD_FLEX_CTX_QW1_FLEX1_M 0xFF
+};
+#endif /* _IDPF_LAN_TXRX_H_ */
diff --git a/drivers/common/idpf/base/idpf_lan_vf_regs.h b/drivers/common/idpf/base/idpf_lan_vf_regs.h
new file mode 100644
index 0000000000..9cd4f757d9
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_lan_vf_regs.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LAN_VF_REGS_H_
+#define _IDPF_LAN_VF_REGS_H_
+
+
+/* Reset */
+#define VFGEN_RSTAT 0x00008800
+#define VFGEN_RSTAT_VFR_STATE_S 0
+#define VFGEN_RSTAT_VFR_STATE_M MAKEMASK(0x3, VFGEN_RSTAT_VFR_STATE_S)
+
+/* Control(VF Mailbox) Queue */
+#define VF_BASE 0x00006000
+
+#define VF_ATQBAL (VF_BASE + 0x1C00)
+#define VF_ATQBAH (VF_BASE + 0x1800)
+#define VF_ATQLEN (VF_BASE + 0x0800)
+#define VF_ATQLEN_ATQLEN_S 0
+#define VF_ATQLEN_ATQLEN_M MAKEMASK(0x3FF, VF_ATQLEN_ATQLEN_S)
+#define VF_ATQLEN_ATQVFE_S 28
+#define VF_ATQLEN_ATQVFE_M BIT(VF_ATQLEN_ATQVFE_S)
+#define VF_ATQLEN_ATQOVFL_S 29
+#define VF_ATQLEN_ATQOVFL_M BIT(VF_ATQLEN_ATQOVFL_S)
+#define VF_ATQLEN_ATQCRIT_S 30
+#define VF_ATQLEN_ATQCRIT_M BIT(VF_ATQLEN_ATQCRIT_S)
+#define VF_ATQLEN_ATQENABLE_S 31
+#define VF_ATQLEN_ATQENABLE_M BIT(VF_ATQLEN_ATQENABLE_S)
+#define VF_ATQH (VF_BASE + 0x0400)
+#define VF_ATQH_ATQH_S 0
+#define VF_ATQH_ATQH_M MAKEMASK(0x3FF, VF_ATQH_ATQH_S)
+#define VF_ATQT (VF_BASE + 0x2400)
+
+#define VF_ARQBAL (VF_BASE + 0x0C00)
+#define VF_ARQBAH (VF_BASE)
+#define VF_ARQLEN (VF_BASE + 0x2000)
+#define VF_ARQLEN_ARQLEN_S 0
+#define VF_ARQLEN_ARQLEN_M MAKEMASK(0x3FF, VF_ARQLEN_ARQLEN_S)
+#define VF_ARQLEN_ARQVFE_S 28
+#define VF_ARQLEN_ARQVFE_M BIT(VF_ARQLEN_ARQVFE_S)
+#define VF_ARQLEN_ARQOVFL_S 29
+#define VF_ARQLEN_ARQOVFL_M BIT(VF_ARQLEN_ARQOVFL_S)
+#define VF_ARQLEN_ARQCRIT_S 30
+#define VF_ARQLEN_ARQCRIT_M BIT(VF_ARQLEN_ARQCRIT_S)
+#define VF_ARQLEN_ARQENABLE_S 31
+#define VF_ARQLEN_ARQENABLE_M BIT(VF_ARQLEN_ARQENABLE_S)
+#define VF_ARQH (VF_BASE + 0x1400)
+#define VF_ARQH_ARQH_S 0
+#define VF_ARQH_ARQH_M MAKEMASK(0x1FFF, VF_ARQH_ARQH_S)
+#define VF_ARQT (VF_BASE + 0x1000)
+
+/* Transmit queues */
+#define VF_QTX_TAIL_BASE 0x00000000
+#define VF_QTX_TAIL(_QTX) (VF_QTX_TAIL_BASE + (_QTX) * 0x4)
+#define VF_QTX_TAIL_EXT_BASE 0x00040000
+#define VF_QTX_TAIL_EXT(_QTX) (VF_QTX_TAIL_EXT_BASE + ((_QTX) * 4))
+
+/* Receive queues */
+#define VF_QRX_TAIL_BASE 0x00002000
+#define VF_QRX_TAIL(_QRX) (VF_QRX_TAIL_BASE + ((_QRX) * 4))
+#define VF_QRX_TAIL_EXT_BASE 0x00050000
+#define VF_QRX_TAIL_EXT(_QRX) (VF_QRX_TAIL_EXT_BASE + ((_QRX) * 4))
+#define VF_QRXB_TAIL_BASE 0x00060000
+#define VF_QRXB_TAIL(_QRX) (VF_QRXB_TAIL_BASE + ((_QRX) * 4))
+
+/* Interrupts */
+#define VF_INT_DYN_CTL0 0x00005C00
+#define VF_INT_DYN_CTL0_INTENA_S 0
+#define VF_INT_DYN_CTL0_INTENA_M BIT(VF_INT_DYN_CTL0_INTENA_S)
+#define VF_INT_DYN_CTL0_ITR_INDX_S 3
+#define VF_INT_DYN_CTL0_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTL0_ITR_INDX_S)
+#define VF_INT_DYN_CTLN(_INT) (0x00003800 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_EXT(_INT) (0x00070000 + ((_INT) * 4))
+#define VF_INT_DYN_CTLN_INTENA_S 0
+#define VF_INT_DYN_CTLN_INTENA_M BIT(VF_INT_DYN_CTLN_INTENA_S)
+#define VF_INT_DYN_CTLN_CLEARPBA_S 1
+#define VF_INT_DYN_CTLN_CLEARPBA_M BIT(VF_INT_DYN_CTLN_CLEARPBA_S)
+#define VF_INT_DYN_CTLN_SWINT_TRIG_S 2
+#define VF_INT_DYN_CTLN_SWINT_TRIG_M BIT(VF_INT_DYN_CTLN_SWINT_TRIG_S)
+#define VF_INT_DYN_CTLN_ITR_INDX_S 3
+#define VF_INT_DYN_CTLN_ITR_INDX_M MAKEMASK(0x3, VF_INT_DYN_CTLN_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_INTERVAL_S 5
+#define VF_INT_DYN_CTLN_INTERVAL_M BIT(VF_INT_DYN_CTLN_INTERVAL_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S 24
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_ENA_S)
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_S 25
+#define VF_INT_DYN_CTLN_SW_ITR_INDX_M BIT(VF_INT_DYN_CTLN_SW_ITR_INDX_S)
+#define VF_INT_DYN_CTLN_WB_ON_ITR_S 30
+#define VF_INT_DYN_CTLN_WB_ON_ITR_M BIT(VF_INT_DYN_CTLN_WB_ON_ITR_S)
+#define VF_INT_DYN_CTLN_INTENA_MSK_S 31
+#define VF_INT_DYN_CTLN_INTENA_MSK_M BIT(VF_INT_DYN_CTLN_INTENA_MSK_S)
+#define VF_INT_ITR0(_i) (0x00004C00 + ((_i) * 4))
+#define VF_INT_ITRN_V2(_i, _reg_start) ((_reg_start) + (((_i)) * 4))
+#define VF_INT_ITRN(_i, _INT) (0x00002800 + ((_i) * 4) + ((_INT) * 0x40))
+#define VF_INT_ITRN_64(_i, _INT) (0x00002C00 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_2K(_i, _INT) (0x00072000 + ((_i) * 4) + ((_INT) * 0x100))
+#define VF_INT_ITRN_MAX_INDEX 2
+#define VF_INT_ITRN_INTERVAL_S 0
+#define VF_INT_ITRN_INTERVAL_M MAKEMASK(0xFFF, VF_INT_ITRN_INTERVAL_S)
+#define VF_INT_PBA_CLEAR 0x00008900
+
+#define VF_INT_ICR0_ENA1 0x00005000
+#define VF_INT_ICR0_ENA1_ADMINQ_S 30
+#define VF_INT_ICR0_ENA1_ADMINQ_M BIT(VF_INT_ICR0_ENA1_ADMINQ_S)
+#define VF_INT_ICR0_ENA1_RSVD_S 31
+#define VF_INT_ICR01 0x00004800
+#define VF_QF_HENA(_i) (0x0000C400 + ((_i) * 4))
+#define VF_QF_HENA_MAX_INDX 1
+#define VF_QF_HKEY(_i) (0x0000CC00 + ((_i) * 4))
+#define VF_QF_HKEY_MAX_INDX 12
+#define VF_QF_HLUT(_i) (0x0000D000 + ((_i) * 4))
+#define VF_QF_HLUT_MAX_INDX 15
+#endif
diff --git a/drivers/common/idpf/base/idpf_osdep.h b/drivers/common/idpf/base/idpf_osdep.h
new file mode 100644
index 0000000000..99ae9cf60a
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_osdep.h
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_OSDEP_H_
+#define _IDPF_OSDEP_H_
+
+#include <string.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <sys/queue.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_byteorder.h>
+#include <rte_cycles.h>
+#include <rte_spinlock.h>
+#include <rte_log.h>
+#include <rte_random.h>
+#include <rte_io.h>
+
+#define INLINE inline
+#define STATIC static
+
+typedef uint8_t u8;
+typedef int8_t s8;
+typedef uint16_t u16;
+typedef int16_t s16;
+typedef uint32_t u32;
+typedef int32_t s32;
+typedef uint64_t u64;
+typedef uint64_t s64;
+
+typedef struct idpf_lock idpf_lock;
+
+#define __iomem
+#define hw_dbg(hw, S, A...) do {} while (0)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)(n))
+#define low_16_bits(x) ((x) & 0xFFFF)
+#define high_16_bits(x) (((x) & 0xFFFF0000) >> 16)
+
+#ifndef ETH_ADDR_LEN
+#define ETH_ADDR_LEN 6
+#endif
+
+#ifndef __le16
+#define __le16 uint16_t
+#endif
+#ifndef __le32
+#define __le32 uint32_t
+#endif
+#ifndef __le64
+#define __le64 uint64_t
+#endif
+#ifndef __be16
+#define __be16 uint16_t
+#endif
+#ifndef __be32
+#define __be32 uint32_t
+#endif
+#ifndef __be64
+#define __be64 uint64_t
+#endif
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) RTE_BIT64(a)
+#endif
+
+#ifndef BIT
+#define BIT(a) RTE_BIT32(a)
+#endif
+
+#define FALSE 0
+#define TRUE 1
+#define false 0
+#define true 1
+
+/* Avoid macro redefinition warning on Windows */
+#ifdef RTE_EXEC_ENV_WINDOWS
+#ifdef min
+#undef min
+#endif
+#ifdef max
+#undef max
+#endif
+#endif
+
+#define min(a, b) RTE_MIN(a, b)
+#define max(a, b) RTE_MAX(a, b)
+
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+#define FIELD_SIZEOF(t, f) (sizeof(((t *)0)->(f)))
+#define MAKEMASK(m, s) ((m) << (s))
+
+extern int idpf_common_logger;
+
+#define DEBUGOUT(S) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S)
+#define DEBUGOUT2(S, A...) rte_log(RTE_LOG_DEBUG, idpf_common_logger, S, ##A)
+#define DEBUGFUNC(F) DEBUGOUT(F "\n")
+
+#define idpf_debug(h, m, s, ...) \
+ do { \
+ if (((m) & (h)->debug_mask)) \
+ PMD_DRV_LOG_RAW(DEBUG, "idpf %02x.%x " s, \
+ (h)->bus.device, (h)->bus.func, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define idpf_info(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_warn(hw, fmt, args...) idpf_debug(hw, IDPF_DBG_ALL, fmt, ##args)
+#define idpf_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ do { \
+ struct idpf_hw *hw_l = hw; \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < len_l; i += 8) \
+ idpf_debug(hw_l, type, \
+ "0x%04X 0x%016"PRIx64"\n", \
+ i, *((u64 *)((buf_l) + i))); \
+ } while (0)
+#define idpf_snprintf snprintf
+#ifndef SNPRINTF
+#define SNPRINTF idpf_snprintf
+#endif
+
+#define IDPF_PCI_REG(reg) rte_read32(reg)
+#define IDPF_PCI_REG_ADDR(a, reg) \
+ ((volatile uint32_t *)((char *)(a)->hw_addr + (reg)))
+#define IDPF_PCI_REG64(reg) rte_read64(reg)
+#define IDPF_PCI_REG_ADDR64(a, reg) \
+ ((volatile uint64_t *)((char *)(a)->hw_addr + (reg)))
+
+#define idpf_wmb() rte_io_wmb()
+#define idpf_rmb() rte_io_rmb()
+#define idpf_mb() rte_io_mb()
+
+static inline uint32_t idpf_read_addr(volatile void *addr)
+{
+ return rte_le_to_cpu_32(IDPF_PCI_REG(addr));
+}
+
+static inline uint64_t idpf_read_addr64(volatile void *addr)
+{
+ return rte_le_to_cpu_64(IDPF_PCI_REG64(addr));
+}
+
+#define IDPF_PCI_REG_WRITE(reg, value) \
+ rte_write32((rte_cpu_to_le_32(value)), reg)
+
+#define IDPF_PCI_REG_WRITE64(reg, value) \
+ rte_write64((rte_cpu_to_le_64(value)), reg)
+
+#define IDPF_READ_REG(hw, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((hw), (reg)))
+#define IDPF_WRITE_REG(hw, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((hw), (reg)), (value))
+
+#define rd32(a, reg) idpf_read_addr(IDPF_PCI_REG_ADDR((a), (reg)))
+#define wr32(a, reg, value) \
+ IDPF_PCI_REG_WRITE(IDPF_PCI_REG_ADDR((a), (reg)), (value))
+#define div64_long(n, d) ((n) / (d))
+#define rd64(a, reg) idpf_read_addr64(IDPF_PCI_REG_ADDR64((a), (reg)))
+
+#define BITS_PER_BYTE 8
+
+/* memory allocation tracking */
+struct idpf_dma_mem {
+ void *va;
+ u64 pa;
+ u32 size;
+ const void *zone;
+} __rte_packed;
+
+struct idpf_virt_mem {
+ void *va;
+ u32 size;
+} __rte_packed;
+
+#define idpf_malloc(h, s) rte_zmalloc(NULL, s, 0)
+#define idpf_calloc(h, c, s) rte_zmalloc(NULL, (c) * (s), 0)
+#define idpf_free(h, m) rte_free(m)
+
+#define idpf_memset(a, b, c, d) memset((a), (b), (c))
+#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+/* SW spinlock */
+struct idpf_lock {
+ rte_spinlock_t spinlock;
+};
+
+static inline void
+idpf_init_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_init(&sp->spinlock);
+}
+
+static inline void
+idpf_acquire_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_lock(&sp->spinlock);
+}
+
+static inline void
+idpf_release_lock(struct idpf_lock *sp)
+{
+ rte_spinlock_unlock(&sp->spinlock);
+}
+
+static inline void
+idpf_destroy_lock(__rte_unused struct idpf_lock *sp)
+{
+}
+
+struct idpf_hw;
+
+static inline void *
+idpf_alloc_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem, u64 size)
+{
+ const struct rte_memzone *mz = NULL;
+ char z_name[RTE_MEMZONE_NAMESIZE];
+
+ if (!mem)
+ return NULL;
+
+ snprintf(z_name, sizeof(z_name), "idpf_dma_%"PRIu64, rte_rand());
+ mz = rte_memzone_reserve_aligned(z_name, size, SOCKET_ID_ANY,
+ RTE_MEMZONE_IOVA_CONTIG, RTE_PGSIZE_4K);
+ if (!mz)
+ return NULL;
+
+ mem->size = size;
+ mem->va = mz->addr;
+ mem->pa = mz->iova;
+ mem->zone = (const void *)mz;
+ memset(mem->va, 0, size);
+
+ return mem->va;
+}
+
+static inline void
+idpf_free_dma_mem(__rte_unused struct idpf_hw *hw,
+ struct idpf_dma_mem *mem)
+{
+ rte_memzone_free((const struct rte_memzone *)mem->zone);
+ mem->size = 0;
+ mem->va = NULL;
+ mem->pa = 0;
+}
+
+static inline u8
+idpf_hweight8(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline u8
+idpf_hweight32(u32 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 32; i++) {
+ bits += (u8)(num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
+#define DELAY(x) rte_delay_us(x)
+#define idpf_usec_delay(x) rte_delay_us(x)
+#define idpf_msec_delay(x, y) rte_delay_us(1000 * (x))
+#define udelay(x) DELAY(x)
+#define msleep(x) DELAY(1000 * (x))
+#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+
+#ifndef IDPF_DBG_TRACE
+#define IDPF_DBG_TRACE BIT_ULL(0)
+#endif
+
+#ifndef DIVIDE_AND_ROUND_UP
+#define DIVIDE_AND_ROUND_UP(a, b) (((a) + (b) - 1) / (b))
+#endif
+
+#ifndef IDPF_INTEL_VENDOR_ID
+#define IDPF_INTEL_VENDOR_ID 0x8086
+#endif
+
+#ifndef IS_UNICAST_ETHER_ADDR
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+#endif
+
+#ifndef IS_MULTICAST_ETHER_ADDR
+#define IS_MULTICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 1))
+#endif
+
+#ifndef IS_BROADCAST_ETHER_ADDR
+/* Check whether an address is broadcast. */
+#define IS_BROADCAST_ETHER_ADDR(addr) \
+ ((bool)((((u16 *)(addr))[0] == ((u16)0xffff))))
+#endif
+
+#ifndef IS_ZERO_ETHER_ADDR
+#define IS_ZERO_ETHER_ADDR(addr) \
+ (((bool)((((u16 *)(addr))[0] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[1] == ((u16)0x0)))) && \
+ ((bool)((((u16 *)(addr))[2] == ((u16)0x0)))))
+#endif
+
+#ifndef LIST_HEAD_TYPE
+#define LIST_HEAD_TYPE(list_name, type) LIST_HEAD(list_name, type)
+#endif
+
+#ifndef LIST_ENTRY_TYPE
+#define LIST_ENTRY_TYPE(type) LIST_ENTRY(type)
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY_SAFE
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, temp, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#ifndef LIST_FOR_EACH_ENTRY
+#define LIST_FOR_EACH_ENTRY(pos, head, entry_type, list) \
+ LIST_FOREACH(pos, head, list)
+
+#endif
+
+#endif /* _IDPF_OSDEP_H_ */
diff --git a/drivers/common/idpf/base/idpf_prototype.h b/drivers/common/idpf/base/idpf_prototype.h
new file mode 100644
index 0000000000..529b62212d
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_prototype.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_PROTOTYPE_H_
+#define _IDPF_PROTOTYPE_H_
+
+/* Include generic macros and types first */
+#include "idpf_osdep.h"
+#include "idpf_controlq.h"
+#include "idpf_type.h"
+#include "idpf_alloc.h"
+#include "idpf_devids.h"
+#include "idpf_controlq_api.h"
+#include "idpf_lan_pf_regs.h"
+#include "idpf_lan_vf_regs.h"
+#include "idpf_lan_txrx.h"
+#include "virtchnl.h"
+
+#define APF
+
+int idpf_init_hw(struct idpf_hw *hw, struct idpf_ctlq_size ctlq_size);
+int idpf_deinit_hw(struct idpf_hw *hw);
+
+int idpf_clean_arq_element(struct idpf_hw *hw,
+ struct idpf_arq_event_info *e,
+ u16 *events_pending);
+bool idpf_asq_done(struct idpf_hw *hw);
+bool idpf_check_asq_alive(struct idpf_hw *hw);
+
+int idpf_get_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_set_rss_lut(struct idpf_hw *hw, u16 seid, bool pf_lut,
+ u8 *lut, u16 lut_size);
+int idpf_get_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+int idpf_set_rss_key(struct idpf_hw *hw, u16 seid,
+ struct idpf_get_set_rss_key_data *key);
+
+int idpf_set_mac_type(struct idpf_hw *hw);
+
+int idpf_reset(struct idpf_hw *hw);
+int idpf_send_msg_to_cp(struct idpf_hw *hw, enum virtchnl_ops v_opcode,
+ int v_retval, u8 *msg, u16 msglen);
+#endif /* _IDPF_PROTOTYPE_H_ */
diff --git a/drivers/common/idpf/base/idpf_type.h b/drivers/common/idpf/base/idpf_type.h
new file mode 100644
index 0000000000..3b46536287
--- /dev/null
+++ b/drivers/common/idpf/base/idpf_type.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _IDPF_TYPE_H_
+#define _IDPF_TYPE_H_
+
+#include "idpf_controlq.h"
+
+#define UNREFERENCED_XPARAMETER
+#define UNREFERENCED_1PARAMETER(_p)
+#define UNREFERENCED_2PARAMETER(_p, _q)
+#define UNREFERENCED_3PARAMETER(_p, _q, _r)
+#define UNREFERENCED_4PARAMETER(_p, _q, _r, _s)
+#define UNREFERENCED_5PARAMETER(_p, _q, _r, _s, _t)
+
+#define MAKEMASK(m, s) ((m) << (s))
+
+struct idpf_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct idpf_hw_port_stats {
+ /* eth stats collected by the port */
+ struct idpf_eth_stats eth;
+
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_length_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 priority_xon_rx[8]; /* pxonrxc[8] */
+ u64 priority_xoff_rx[8]; /* pxoffrxc[8] */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 priority_xon_tx[8]; /* pxontxc[8] */
+ u64 priority_xoff_tx[8]; /* pxofftxc[8] */
+ u64 priority_xon_2_xoff[8]; /* pxon2offc[8] */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+ u64 mac_short_packet_dropped; /* mspdc */
+ u64 checksum_error; /* xec */
+};
+/* Static buffer size to initialize control queue */
+struct idpf_ctlq_size {
+ u16 asq_buf_size;
+ u16 asq_ring_size;
+ u16 arq_buf_size;
+ u16 arq_ring_size;
+};
+
+/* Temporary definition to compile - TBD if needed */
+struct idpf_arq_event_info {
+ struct idpf_ctlq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+struct idpf_get_set_rss_key_data {
+ u8 standard_rss_key[0x28];
+ u8 extended_hash_key[0xc];
+};
+
+struct idpf_aq_get_phy_abilities_resp {
+ __le32 phy_type;
+};
+
+struct idpf_filter_program_desc {
+ __le32 qid;
+};
+
+#endif /* _IDPF_TYPE_H_ */
diff --git a/drivers/common/idpf/base/meson.build b/drivers/common/idpf/base/meson.build
new file mode 100644
index 0000000000..ce2b5346e4
--- /dev/null
+++ b/drivers/common/idpf/base/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+ 'idpf_common.c',
+ 'idpf_controlq.c',
+ 'idpf_controlq_setup.c',
+)
+
+cflags += ['-Wno-unused-value']
+cflags += ['-Wno-unused-variable']
+cflags += ['-Wno-unused-parameter']
+cflags += ['-Wno-implicit-fallthrough']
+cflags += ['-Wno-strict-aliasing']
\ No newline at end of file
diff --git a/drivers/common/idpf/base/siov_regs.h b/drivers/common/idpf/base/siov_regs.h
new file mode 100644
index 0000000000..3ac4f8f177
--- /dev/null
+++ b/drivers/common/idpf/base/siov_regs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+#ifndef _SIOV_REGS_H_
+#define _SIOV_REGS_H_
+#define VDEV_MBX_START 0x20000 /* Begin at 128KB */
+#define VDEV_MBX_ATQBAL (VDEV_MBX_START + 0x0000)
+#define VDEV_MBX_ATQBAH (VDEV_MBX_START + 0x0004)
+#define VDEV_MBX_ATQLEN (VDEV_MBX_START + 0x0008)
+#define VDEV_MBX_ATQH (VDEV_MBX_START + 0x000C)
+#define VDEV_MBX_ATQT (VDEV_MBX_START + 0x0010)
+#define VDEV_MBX_ARQBAL (VDEV_MBX_START + 0x0014)
+#define VDEV_MBX_ARQBAH (VDEV_MBX_START + 0x0018)
+#define VDEV_MBX_ARQLEN (VDEV_MBX_START + 0x001C)
+#define VDEV_MBX_ARQH (VDEV_MBX_START + 0x0020)
+#define VDEV_MBX_ARQT (VDEV_MBX_START + 0x0024)
+#define VDEV_GET_RSTAT 0x21000 /* 132KB for RSTAT */
+
+/* Begin at offset after 1MB (after 256 4k pages) */
+#define VDEV_QRX_TAIL_START 0x100000
+#define VDEV_QRX_TAIL(_i) (VDEV_QRX_TAIL_START + ((_i) * 0x1000)) /* 2k Rx queues */
+
+/* Begin at offset of 9MB for Rx buffer queue tail register pages */
+#define VDEV_QRX_BUFQ_TAIL_START 0x900000
+/* 2k Rx buffer queues */
+#define VDEV_QRX_BUFQ_TAIL(_i) (VDEV_QRX_BUFQ_TAIL_START + ((_i) * 0x1000))
+
+/* Begin at offset of 17MB for 2k Tx queues */
+#define VDEV_QTX_TAIL_START 0x1100000
+#define VDEV_QTX_TAIL(_i) (VDEV_QTX_TAIL_START + ((_i) * 0x1000)) /* 2k Tx queues */
+
+/* Begin at offset of 25MB for 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL_START 0x1900000
+/* 2k Tx completion queues */
+#define VDEV_QTX_COMPL_TAIL(_i) (VDEV_QTX_COMPL_TAIL_START + ((_i) * 0x1000))
+
+#define VDEV_INT_DYN_CTL01 0x2100000 /* Begin at offset 33MB */
+
+/* Begin at offset of 33MB + 4k to accommodate CTL01 register */
+#define VDEV_INT_DYN_START (VDEV_INT_DYN_CTL01 + 0x1000)
+#define VDEV_INT_DYN_CTL(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000))
+#define VDEV_INT_ITR_0(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x04)
+#define VDEV_INT_ITR_1(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x08)
+#define VDEV_INT_ITR_2(_i) (VDEV_INT_DYN_START + ((_i) * 0x1000) + 0x0C)
+
+/* Next offset to begin at 42MB (0x2A00000) */
+#endif /* _SIOV_REGS_H_ */
diff --git a/drivers/common/idpf/base/virtchnl.h b/drivers/common/idpf/base/virtchnl.h
new file mode 100644
index 0000000000..ea798e3971
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl.h
@@ -0,0 +1,2866 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_H_
+#define _VIRTCHNL_H_
+
+/* Description:
+ * This header file describes the Virtual Function (VF) - Physical Function
+ * (PF) communication protocol used by the drivers for all devices starting
+ * from our 40G product line
+ *
+ * Admin queue buffer usage:
+ * desc->opcode is always aqc_opc_send_msg_to_pf
+ * flags, retval, datalen, and data addr are all used normally.
+ * The Firmware copies the cookie fields when sending messages between the
+ * PF and VF, but uses all other fields internally. Due to this limitation,
+ * we must send all messages as "indirect", i.e. using an external buffer.
+ *
+ * All the VSI indexes are relative to the VF. Each VF can have maximum of
+ * three VSIs. All the queue indexes are relative to the VSI. Each VF can
+ * have a maximum of sixteen queues for all of its VSIs.
+ *
+ * The PF is required to return a status code in v_retval for all messages
+ * except RESET_VF, which does not require any response. The returned value
+ * is of virtchnl_status_code type, defined here.
+ *
+ * In general, VF driver initialization should roughly follow the order of
+ * these opcodes. The VF driver must first validate the API version of the
+ * PF driver, then request a reset, then get resources, then configure
+ * queues and interrupts. After these operations are complete, the VF
+ * driver may start its queues, optionally add MAC and VLAN filters, and
+ * process traffic.
+ */
+
+/* START GENERIC DEFINES
+ * Need to ensure the following enums and defines hold the same meaning and
+ * value in current and future projects
+ */
+
+#define VIRTCHNL_ETH_LENGTH_OF_ADDRESS 6
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL_CHECK_STRUCT_LEN(n, X) enum virtchnl_static_assert_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL_CHECK_UNION_LEN(n, X) enum virtchnl_static_asset_enum_##X \
+ { virtchnl_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+enum virtchnl_status_code {
+ VIRTCHNL_STATUS_SUCCESS = 0,
+ VIRTCHNL_STATUS_ERR_PARAM = -5,
+ VIRTCHNL_STATUS_ERR_NO_MEMORY = -18,
+ VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH = -38,
+ VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR = -39,
+ VIRTCHNL_STATUS_ERR_INVALID_VF_ID = -40,
+ VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR = -53,
+ VIRTCHNL_STATUS_ERR_NOT_SUPPORTED = -64,
+};
+
+/* Backward compatibility */
+#define VIRTCHNL_ERR_PARAM VIRTCHNL_STATUS_ERR_PARAM
+#define VIRTCHNL_STATUS_NOT_SUPPORTED VIRTCHNL_STATUS_ERR_NOT_SUPPORTED
+
+#define VIRTCHNL_LINK_SPEED_2_5GB_SHIFT 0x0
+#define VIRTCHNL_LINK_SPEED_100MB_SHIFT 0x1
+#define VIRTCHNL_LINK_SPEED_1000MB_SHIFT 0x2
+#define VIRTCHNL_LINK_SPEED_10GB_SHIFT 0x3
+#define VIRTCHNL_LINK_SPEED_40GB_SHIFT 0x4
+#define VIRTCHNL_LINK_SPEED_20GB_SHIFT 0x5
+#define VIRTCHNL_LINK_SPEED_25GB_SHIFT 0x6
+#define VIRTCHNL_LINK_SPEED_5GB_SHIFT 0x7
+
+enum virtchnl_link_speed {
+ VIRTCHNL_LINK_SPEED_UNKNOWN = 0,
+ VIRTCHNL_LINK_SPEED_100MB = BIT(VIRTCHNL_LINK_SPEED_100MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_1GB = BIT(VIRTCHNL_LINK_SPEED_1000MB_SHIFT),
+ VIRTCHNL_LINK_SPEED_10GB = BIT(VIRTCHNL_LINK_SPEED_10GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_40GB = BIT(VIRTCHNL_LINK_SPEED_40GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_20GB = BIT(VIRTCHNL_LINK_SPEED_20GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_25GB = BIT(VIRTCHNL_LINK_SPEED_25GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_2_5GB = BIT(VIRTCHNL_LINK_SPEED_2_5GB_SHIFT),
+ VIRTCHNL_LINK_SPEED_5GB = BIT(VIRTCHNL_LINK_SPEED_5GB_SHIFT),
+};
+
+/* for hsplit_0 field of Rx HMC context */
+/* deprecated with AVF 1.0 */
+enum virtchnl_rx_hsplit {
+ VIRTCHNL_RX_HSPLIT_NO_SPLIT = 0,
+ VIRTCHNL_RX_HSPLIT_SPLIT_L2 = 1,
+ VIRTCHNL_RX_HSPLIT_SPLIT_IP = 2,
+ VIRTCHNL_RX_HSPLIT_SPLIT_TCP_UDP = 4,
+ VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
+};
+
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
+/* END GENERIC DEFINES */
+
+/* Opcodes for VF-PF communication. These are placed in the v_opcode field
+ * of the virtchnl_msg structure.
+ */
+enum virtchnl_ops {
+/* The PF sends status change events to VFs using
+ * the VIRTCHNL_OP_EVENT opcode.
+ * VFs send requests to the PF using the other ops.
+ * Use of "advanced opcode" features must be negotiated as part of capabilities
+ * exchange and are not considered part of base mode feature set.
+ *
+ */
+ VIRTCHNL_OP_UNKNOWN = 0,
+ VIRTCHNL_OP_VERSION = 1, /* must ALWAYS be 1 */
+ VIRTCHNL_OP_RESET_VF = 2,
+ VIRTCHNL_OP_GET_VF_RESOURCES = 3,
+ VIRTCHNL_OP_CONFIG_TX_QUEUE = 4,
+ VIRTCHNL_OP_CONFIG_RX_QUEUE = 5,
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES = 6,
+ VIRTCHNL_OP_CONFIG_IRQ_MAP = 7,
+ VIRTCHNL_OP_ENABLE_QUEUES = 8,
+ VIRTCHNL_OP_DISABLE_QUEUES = 9,
+ VIRTCHNL_OP_ADD_ETH_ADDR = 10,
+ VIRTCHNL_OP_DEL_ETH_ADDR = 11,
+ VIRTCHNL_OP_ADD_VLAN = 12,
+ VIRTCHNL_OP_DEL_VLAN = 13,
+ VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE = 14,
+ VIRTCHNL_OP_GET_STATS = 15,
+ VIRTCHNL_OP_RSVD = 16,
+ VIRTCHNL_OP_EVENT = 17, /* must ALWAYS be 17 */
+ /* opcode 19 is reserved */
+ /* opcodes 20, 21, and 22 are reserved */
+ VIRTCHNL_OP_CONFIG_RSS_KEY = 23,
+ VIRTCHNL_OP_CONFIG_RSS_LUT = 24,
+ VIRTCHNL_OP_GET_RSS_HENA_CAPS = 25,
+ VIRTCHNL_OP_SET_RSS_HENA = 26,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING = 27,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING = 28,
+ VIRTCHNL_OP_REQUEST_QUEUES = 29,
+ VIRTCHNL_OP_ENABLE_CHANNELS = 30,
+ VIRTCHNL_OP_DISABLE_CHANNELS = 31,
+ VIRTCHNL_OP_ADD_CLOUD_FILTER = 32,
+ VIRTCHNL_OP_DEL_CLOUD_FILTER = 33,
+ /* opcode 34 is reserved */
+ /* opcodes 38, 39, 40, 41, 42 and 43 are reserved */
+ /* opcode 44 is reserved */
+ VIRTCHNL_OP_ADD_RSS_CFG = 45,
+ VIRTCHNL_OP_DEL_RSS_CFG = 46,
+ VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
+ VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
+ VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
+ VIRTCHNL_OP_ADD_VLAN_V2 = 52,
+ VIRTCHNL_OP_DEL_VLAN_V2 = 53,
+ VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 = 54,
+ VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
+ VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
+ VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2 = 58,
+ VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 = 59,
+ VIRTCHNL_OP_1588_PTP_GET_CAPS = 60,
+ VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
+ VIRTCHNL_OP_1588_PTP_SET_TIME = 62,
+ VIRTCHNL_OP_1588_PTP_ADJ_TIME = 63,
+ VIRTCHNL_OP_1588_PTP_ADJ_FREQ = 64,
+ VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP = 65,
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP = 67,
+ VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS = 68,
+ VIRTCHNL_OP_1588_PTP_SET_PIN_CFG = 69,
+ VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP = 70,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
+ VIRTCHNL_OP_MAX,
+};
+
+static inline const char *virtchnl_op_str(enum virtchnl_ops v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL_OP_UNKNOWN:
+ return "VIRTCHNL_OP_UNKNOWN";
+ case VIRTCHNL_OP_VERSION:
+ return "VIRTCHNL_OP_VERSION";
+ case VIRTCHNL_OP_RESET_VF:
+ return "VIRTCHNL_OP_RESET_VF";
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ return "VIRTCHNL_OP_GET_VF_RESOURCES";
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_TX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ return "VIRTCHNL_OP_CONFIG_RX_QUEUE";
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ return "VIRTCHNL_OP_CONFIG_VSI_QUEUES";
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ return "VIRTCHNL_OP_CONFIG_IRQ_MAP";
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ return "VIRTCHNL_OP_ENABLE_QUEUES";
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ return "VIRTCHNL_OP_DISABLE_QUEUES";
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ return "VIRTCHNL_OP_ADD_ETH_ADDR";
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ return "VIRTCHNL_OP_DEL_ETH_ADDR";
+ case VIRTCHNL_OP_ADD_VLAN:
+ return "VIRTCHNL_OP_ADD_VLAN";
+ case VIRTCHNL_OP_DEL_VLAN:
+ return "VIRTCHNL_OP_DEL_VLAN";
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ return "VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE";
+ case VIRTCHNL_OP_GET_STATS:
+ return "VIRTCHNL_OP_GET_STATS";
+ case VIRTCHNL_OP_RSVD:
+ return "VIRTCHNL_OP_RSVD";
+ case VIRTCHNL_OP_EVENT:
+ return "VIRTCHNL_OP_EVENT";
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ return "VIRTCHNL_OP_CONFIG_RSS_KEY";
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ return "VIRTCHNL_OP_CONFIG_RSS_LUT";
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ return "VIRTCHNL_OP_GET_RSS_HENA_CAPS";
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ return "VIRTCHNL_OP_SET_RSS_HENA";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING";
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ return "VIRTCHNL_OP_REQUEST_QUEUES";
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ return "VIRTCHNL_OP_ENABLE_CHANNELS";
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ return "VIRTCHNL_OP_DISABLE_CHANNELS";
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ return "VIRTCHNL_OP_ADD_CLOUD_FILTER";
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ return "VIRTCHNL_OP_DEL_CLOUD_FILTER";
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ return "VIRTCHNL_OP_ADD_RSS_CFG";
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ return "VIRTCHNL_OP_DEL_RSS_CFG";
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ return "VIRTCHNL_OP_ADD_FDIR_FILTER";
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ return "VIRTCHNL_OP_DEL_FDIR_FILTER";
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ return "VIRTCHNL_OP_GET_MAX_RSS_QREGION";
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ return "VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS";
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ return "VIRTCHNL_OP_ADD_VLAN_V2";
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ return "VIRTCHNL_OP_DEL_VLAN_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2";
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ return "VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2";
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ return "VIRTCHNL_OP_1588_PTP_GET_CAPS";
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_GET_TIME";
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ return "VIRTCHNL_OP_1588_PTP_SET_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_TIME";
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ return "VIRTCHNL_OP_1588_PTP_ADJ_FREQ";
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP";
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ return "VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS";
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ return "VIRTCHNL_OP_1588_PTP_SET_PIN_CFG";
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ return "VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP";
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_ENABLE_QUEUES_V2";
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ return "VIRTCHNL_OP_DISABLE_QUEUES_V2";
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL_OP_MAX:
+ return "VIRTCHNL_OP_MAX";
+ default:
+ return "Unsupported (update virtchnl.h)";
+ }
+}
+
+static inline const char *virtchnl_stat_str(enum virtchnl_status_code v_status)
+{
+ switch (v_status) {
+ case VIRTCHNL_STATUS_SUCCESS:
+ return "VIRTCHNL_STATUS_SUCCESS";
+ case VIRTCHNL_STATUS_ERR_PARAM:
+ return "VIRTCHNL_STATUS_ERR_PARAM";
+ case VIRTCHNL_STATUS_ERR_NO_MEMORY:
+ return "VIRTCHNL_STATUS_ERR_NO_MEMORY";
+ case VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH:
+ return "VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH";
+ case VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR:
+ return "VIRTCHNL_STATUS_ERR_CQP_COMPL_ERROR";
+ case VIRTCHNL_STATUS_ERR_INVALID_VF_ID:
+ return "VIRTCHNL_STATUS_ERR_INVALID_VF_ID";
+ case VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR:
+ return "VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR";
+ case VIRTCHNL_STATUS_ERR_NOT_SUPPORTED:
+ return "VIRTCHNL_STATUS_ERR_NOT_SUPPORTED";
+ default:
+ return "Unknown status code (update virtchnl.h)";
+ }
+}
+
+/* Virtual channel message descriptor. This overlays the admin queue
+ * descriptor. All other data is passed in external buffers.
+ */
+
+struct virtchnl_msg {
+ u8 pad[8]; /* AQ flags/opcode/len/retval fields */
+
+ /* avoid confusion with desc->opcode */
+ enum virtchnl_ops v_opcode;
+
+ /* ditto for desc->retval */
+ enum virtchnl_status_code v_retval;
+ u32 vfid; /* used by PF when sending to VF */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(20, virtchnl_msg);
+
+/* Message descriptions and data structures. */
+
+/* VIRTCHNL_OP_VERSION
+ * VF posts its version number to the PF. PF responds with its version number
+ * in the same format, along with a return code.
+ * Reply from PF has its major/minor versions also in param0 and param1.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This enum element MUST always be specified as == 1, regardless of other
+ * changes in the API. The PF must always respond to this message without
+ * error regardless of version mismatch.
+ */
+#define VIRTCHNL_VERSION_MAJOR 1
+#define VIRTCHNL_VERSION_MINOR 1
+#define VIRTCHNL_VERSION_MAJOR_2 2
+#define VIRTCHNL_VERSION_MINOR_0 0
+#define VIRTCHNL_VERSION_MINOR_NO_VF_CAPS 0
+
+struct virtchnl_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_version_info);
+
+#define VF_IS_V10(_ver) (((_ver)->major == 1) && ((_ver)->minor == 0))
+#define VF_IS_V11(_ver) (((_ver)->major == 1) && ((_ver)->minor == 1))
+#define VF_IS_V20(_ver) (((_ver)->major == 2) && ((_ver)->minor == 0))
+
+/* VIRTCHNL_OP_RESET_VF
+ * VF sends this request to PF with no parameters
+ * PF does NOT respond! VF driver must delay then poll VFGEN_RSTAT register
+ * until reset completion is indicated. The admin queue must be reinitialized
+ * after this operation.
+ *
+ * When reset is complete, PF must ensure that all queues in all VSIs associated
+ * with the VF are stopped, all queue configurations in the HMC are set to 0,
+ * and all MAC and VLAN filters (except the default MAC address) on all VSIs
+ * are cleared.
+ */
+
+/* VSI types that use VIRTCHNL interface for VF-PF communication. VSI_SRIOV
+ * vsi_type should always be 6 for backward compatibility. Add other fields
+ * as needed.
+ */
+enum virtchnl_vsi_type {
+ VIRTCHNL_VSI_TYPE_INVALID = 0,
+ VIRTCHNL_VSI_SRIOV = 6,
+};
+
+/* VIRTCHNL_OP_GET_VF_RESOURCES
+ * Version 1.0 VF sends this request to PF with no parameters
+ * Version 1.1 VF sends this request to PF with u32 bitmap of its capabilities
+ * PF responds with an indirect message containing
+ * virtchnl_vf_resource and one or more
+ * virtchnl_vsi_resource structures.
+ */
+
+struct virtchnl_vsi_resource {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+
+ /* see enum virtchnl_vsi_type */
+ s32 vsi_type;
+ u16 qset_handle;
+ u8 default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
+
+/* VF capability flags
+ * VIRTCHNL_VF_OFFLOAD_L2 flag is inclusive of base mode L2 offloads including
+ * TX/RX Checksum offloading and TSO for non-tunnelled packets.
+ */
+#define VIRTCHNL_VF_OFFLOAD_L2 BIT(0)
+#define VIRTCHNL_VF_OFFLOAD_IWARP BIT(1)
+#define VIRTCHNL_VF_CAP_RDMA VIRTCHNL_VF_OFFLOAD_IWARP
+#define VIRTCHNL_VF_OFFLOAD_RSS_AQ BIT(3)
+#define VIRTCHNL_VF_OFFLOAD_RSS_REG BIT(4)
+#define VIRTCHNL_VF_OFFLOAD_WB_ON_ITR BIT(5)
+#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
+/* used to negotiate communicating link speeds in Mbps */
+#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+ /* BIT(8) is reserved */
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
+#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
+#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
+#define VIRTCHNL_VF_OFFLOAD_VLAN BIT(16)
+#define VIRTCHNL_VF_OFFLOAD_RX_POLLING BIT(17)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PCTYPE_V2 BIT(18)
+#define VIRTCHNL_VF_OFFLOAD_RSS_PF BIT(19)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP BIT(20)
+#define VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM BIT(21)
+#define VIRTCHNL_VF_OFFLOAD_RX_ENCAP_CSUM BIT(22)
+#define VIRTCHNL_VF_OFFLOAD_ADQ BIT(23)
+#define VIRTCHNL_VF_OFFLOAD_ADQ_V2 BIT(24)
+#define VIRTCHNL_VF_OFFLOAD_USO BIT(25)
+ /* BIT(26) is reserved */
+#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
+#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
+ /* BIT(30) is reserved */
+#define VIRTCHNL_VF_CAP_PTP BIT(31)
+
+#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
+ VIRTCHNL_VF_OFFLOAD_VLAN | \
+ VIRTCHNL_VF_OFFLOAD_RSS_PF)
+
+struct virtchnl_vf_resource {
+ u16 num_vsis;
+ u16 num_queue_pairs;
+ u16 max_vectors;
+ u16 max_mtu;
+
+ u32 vf_cap_flags;
+ u32 rss_key_size;
+ u32 rss_lut_size;
+
+ struct virtchnl_vsi_resource vsi_res[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_vf_resource);
+
+/* VIRTCHNL_OP_CONFIG_TX_QUEUE
+ * VF sends this message to set up parameters for one TX queue.
+ * External data buffer contains one instance of virtchnl_txq_info.
+ * PF configures requested queue and returns a status code.
+ */
+
+/* Tx queue config info */
+struct virtchnl_txq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u16 ring_len; /* number of descriptors, multiple of 8 */
+ u16 headwb_enabled; /* deprecated with AVF 1.0 */
+ u64 dma_ring_addr;
+ u64 dma_headwb_addr; /* deprecated with AVF 1.0 */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_txq_info);
+
+/* RX descriptor IDs (range from 0 to 63) */
+enum virtchnl_rx_desc_ids {
+ VIRTCHNL_RXDID_0_16B_BASE = 0,
+ VIRTCHNL_RXDID_1_32B_BASE = 1,
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC = 2,
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW = 3,
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB = 4,
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL = 5,
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2 = 6,
+ VIRTCHNL_RXDID_7_HW_RSVD = 7,
+ /* 8 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC = 16,
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN = 17,
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4 = 18,
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6 = 19,
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW = 20,
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP = 21,
+ /* 22 through 63 are reserved */
+};
+
+/* RX descriptor ID bitmasks */
+enum virtchnl_rx_desc_id_bitmasks {
+ VIRTCHNL_RXDID_0_16B_BASE_M = BIT(VIRTCHNL_RXDID_0_16B_BASE),
+ VIRTCHNL_RXDID_1_32B_BASE_M = BIT(VIRTCHNL_RXDID_1_32B_BASE),
+ VIRTCHNL_RXDID_2_FLEX_SQ_NIC_M = BIT(VIRTCHNL_RXDID_2_FLEX_SQ_NIC),
+ VIRTCHNL_RXDID_3_FLEX_SQ_SW_M = BIT(VIRTCHNL_RXDID_3_FLEX_SQ_SW),
+ VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB_M = BIT(VIRTCHNL_RXDID_4_FLEX_SQ_NIC_VEB),
+ VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL_M = BIT(VIRTCHNL_RXDID_5_FLEX_SQ_NIC_ACL),
+ VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2_M = BIT(VIRTCHNL_RXDID_6_FLEX_SQ_NIC_2),
+ VIRTCHNL_RXDID_7_HW_RSVD_M = BIT(VIRTCHNL_RXDID_7_HW_RSVD),
+ /* 9 through 15 are reserved */
+ VIRTCHNL_RXDID_16_COMMS_GENERIC_M = BIT(VIRTCHNL_RXDID_16_COMMS_GENERIC),
+ VIRTCHNL_RXDID_17_COMMS_AUX_VLAN_M = BIT(VIRTCHNL_RXDID_17_COMMS_AUX_VLAN),
+ VIRTCHNL_RXDID_18_COMMS_AUX_IPV4_M = BIT(VIRTCHNL_RXDID_18_COMMS_AUX_IPV4),
+ VIRTCHNL_RXDID_19_COMMS_AUX_IPV6_M = BIT(VIRTCHNL_RXDID_19_COMMS_AUX_IPV6),
+ VIRTCHNL_RXDID_20_COMMS_AUX_FLOW_M = BIT(VIRTCHNL_RXDID_20_COMMS_AUX_FLOW),
+ VIRTCHNL_RXDID_21_COMMS_AUX_TCP_M = BIT(VIRTCHNL_RXDID_21_COMMS_AUX_TCP),
+ /* 22 through 63 are reserved */
+};
+
+/* virtchnl_rxq_info_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_rxq_info structure.
+ */
+enum virtchnl_rxq_info_flags {
+ /* If the VIRTCHNL_PTP_RX_TSTAMP bit of the flag field is set, this is
+ * a request to enable Rx timestamp. Other flag bits are currently
+ * reserved and they may be extended in the future.
+ */
+ VIRTCHNL_PTP_RX_TSTAMP = BIT(0),
+};
+
+/* VIRTCHNL_OP_CONFIG_RX_QUEUE
+ * VF sends this message to set up parameters for one RX queue.
+ * External data buffer contains one instance of virtchnl_rxq_info.
+ * PF configures requested queue and returns a status code. The
+ * crc_disable flag disables CRC stripping on the VF. Setting
+ * the crc_disable flag to 1 will disable CRC stripping for each
+ * queue in the VF where the flag is set. The VIRTCHNL_VF_OFFLOAD_CRC
+ * offload must have been set prior to sending this info or the PF
+ * will ignore the request. This flag should be set the same for
+ * all of the queues for a VF.
+ */
+
+/* Rx queue config info */
+struct virtchnl_rxq_info {
+ u16 vsi_id;
+ u16 queue_id;
+ u32 ring_len; /* number of descriptors, multiple of 32 */
+ u16 hdr_size;
+ u16 splithdr_enabled; /* deprecated with AVF 1.0 */
+ u32 databuffer_size;
+ u32 max_pkt_size;
+ u8 crc_disable;
+ u8 pad1[3];
+ u64 dma_ring_addr;
+
+ /* see enum virtchnl_rx_hsplit; deprecated with AVF 1.0 */
+ s32 rx_split_pos;
+ u32 pad2;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_rxq_info);
+
+/* VIRTCHNL_OP_CONFIG_VSI_QUEUES
+ * VF sends this message to set parameters for active TX and RX queues
+ * associated with the specified VSI.
+ * PF configures queues and returns status.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the VSI, an error is returned and no queues are configured.
+ * NOTE: The VF is not required to configure all queues in a single request.
+ * It may send multiple messages. PF drivers must correctly handle all VF
+ * requests.
+ */
+struct virtchnl_queue_pair_info {
+ /* NOTE: vsi_id and queue_id should be identical for both queues. */
+ struct virtchnl_txq_info txq;
+ struct virtchnl_rxq_info rxq;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(64, virtchnl_queue_pair_info);
+
+struct virtchnl_vsi_queue_config_info {
+ u16 vsi_id;
+ u16 num_queue_pairs;
+ u32 pad;
+ struct virtchnl_queue_pair_info qpair[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_vsi_queue_config_info);
+
+/* VIRTCHNL_OP_REQUEST_QUEUES
+ * VF sends this message to request the PF to allocate additional queues to
+ * this VF. Each VF gets a guaranteed number of queues on init but asking for
+ * additional queues must be negotiated. This is a best effort request as it
+ * is possible the PF does not have enough queues left to support the request.
+ * If the PF cannot support the number requested it will respond with the
+ * maximum number it is able to support. If the request is successful, PF will
+ * then reset the VF to institute required changes.
+ */
+
+/* VF resource request */
+struct virtchnl_vf_res_request {
+ u16 num_queue_pairs;
+};
+
+/* VIRTCHNL_OP_CONFIG_IRQ_MAP
+ * VF uses this message to map vectors to queues.
+ * The rxq_map and txq_map fields are bitmaps used to indicate which queues
+ * are to be associated with the specified vector.
+ * The "other" causes are always mapped to vector 0. The VF may not request
+ * that vector 0 be used for traffic.
+ * PF configures interrupt mapping and returns status.
+ * NOTE: due to hardware requirements, all active queues (both TX and RX)
+ * should be mapped to interrupts, even if the driver intends to operate
+ * only in polling mode. In this case the interrupt may be disabled, but
+ * the ITR timer will still run to trigger writebacks.
+ */
+struct virtchnl_vector_map {
+ u16 vsi_id;
+ u16 vector_id;
+ u16 rxq_map;
+ u16 txq_map;
+ u16 rxitr_idx;
+ u16 txitr_idx;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_vector_map);
+
+struct virtchnl_irq_map_info {
+ u16 num_vectors;
+ struct virtchnl_vector_map vecmap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(14, virtchnl_irq_map_info);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES
+ * VIRTCHNL_OP_DISABLE_QUEUES
+ * VF sends these message to enable or disable TX/RX queue pairs.
+ * The queues fields are bitmaps indicating which queues to act upon.
+ * (Currently, we only support 16 queues per VF, but we make the field
+ * u32 to allow for expansion.)
+ * PF performs requested action and returns status.
+ * NOTE: The VF is not required to enable/disable all queues in a single
+ * request. It may send multiple messages.
+ * PF drivers must correctly handle all VF requests.
+ */
+struct virtchnl_queue_select {
+ u16 vsi_id;
+ u16 pad;
+ u32 rx_queues;
+ u32 tx_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in VIRTCHNL_OP_GET_VF_RESOURCES
+ * then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width
+ * of 6 would inform the VF that the PF supports a maximum RSS queue region
+ * of 64.
+ *
+ * A queue region represents a range of queues that can be used to configure
+ * a RSS LUT. For example, if a VF is given 64 queues, but only a max queue
+ * region size of 16 (i.e. 2^qregion_width = 16) then it will only be able
+ * to configure the RSS LUT with queue indices from 0 to 15. However, other
+ * filters can be used to direct packets to queues >15 via specifying a queue
+ * base/offset and queue region width.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
+/* VIRTCHNL_OP_ADD_ETH_ADDR
+ * VF sends this message in order to add one or more unicast or multicast
+ * address filters for the specified VSI.
+ * PF adds the filters and returns status.
+ */
+
+/* VIRTCHNL_OP_DEL_ETH_ADDR
+ * VF sends this message in order to remove one or more unicast or multicast
+ * filters for the specified VSI.
+ * PF removes the filters and returns status.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_LEGACY
+ * Prior to adding the @type member to virtchnl_ether_addr, there were 2 pad
+ * bytes. Moving forward all VF drivers should not set type to
+ * VIRTCHNL_ETHER_ADDR_LEGACY. This is only here to not break previous/legacy
+ * behavior. The control plane function (i.e. PF) can use a best effort method
+ * of tracking the primary/device unicast in this case, but there is no
+ * guarantee and functionality depends on the implementation of the PF.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_PRIMARY
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL_OP_ADD_ETH_ADDR and
+ * VIRTCHNL_OP_DEL_ETH_ADDR. This allows for the underlying control plane
+ * function (i.e. PF) to accurately track and use this MAC address for
+ * displaying on the host and for VM/function reset.
+ */
+
+/* VIRTCHNL_ETHER_ADDR_EXTRA
+ * All VF drivers should set @type to VIRTCHNL_ETHER_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL_OP_DEL_ETH_ADDR/VIRTCHNL_OP_ADD_ETH_ADDR respectively.
+ */
+struct virtchnl_ether_addr {
+ u8 addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 type;
+#define VIRTCHNL_ETHER_ADDR_LEGACY 0
+#define VIRTCHNL_ETHER_ADDR_PRIMARY 1
+#define VIRTCHNL_ETHER_ADDR_EXTRA 2
+#define VIRTCHNL_ETHER_ADDR_TYPE_MASK 3 /* first two bits of type are valid */
+ u8 pad;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_ether_addr);
+
+struct virtchnl_ether_addr_list {
+ u16 vsi_id;
+ u16 num_elements;
+ struct virtchnl_ether_addr list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_ether_addr_list);
+
+/* VIRTCHNL_OP_ADD_VLAN
+ * VF sends this message to add one or more VLAN tag filters for receives.
+ * PF adds the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+/* VIRTCHNL_OP_DEL_VLAN
+ * VF sends this message to remove one or more VLAN tag filters for receives.
+ * PF removes the filters and returns status.
+ * If a port VLAN is configured by the PF, this operation will return an
+ * error to the VF.
+ */
+
+struct virtchnl_vlan_filter_list {
+ u16 vsi_id;
+ u16 num_elements;
+ u16 vlan_id[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_vlan_filter_list);
+
+/* This enum is used for all of the VIRTCHNL_VF_OFFLOAD_VLAN_V2_CAPS related
+ * structures and opcodes.
+ *
+ * VIRTCHNL_VLAN_UNSUPPORTED - This field is not supported and if a VF driver
+ * populates it the PF should return VIRTCHNL_STATUS_ERR_NOT_SUPPORTED.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 - This field supports 0x8100 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 - This field supports 0x88A8 ethertype.
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 - This field supports 0x9100 ethertype.
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_AND - Used when multiple ethertypes can be supported
+ * by the PF concurrently. For example, if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 AND VIRTCHNL_VLAN_ETHERTYPE_88A8 filters it
+ * would OR the following bits:
+ *
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * The VF would interpret this as VLAN filtering can be supported on both 0x8100
+ * and 0x88A8 VLAN ethertypes.
+ *
+ * VIRTCHNL_ETHERTYPE_XOR - Used when only a single ethertype can be supported
+ * by the PF concurrently. For example if the PF can support
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 XOR VIRTCHNL_VLAN_ETHERTYPE_88A8 stripping
+ * offload it would OR the following bits:
+ *
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * The VF would interpret this as VLAN stripping can be supported on either
+ * 0x8100 or 0x88a8 VLAN ethertypes. So when requesting VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 the specified ethertype will override
+ * the previously set value.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 - Used to tell the VF to insert and/or
+ * strip the VLAN tag using the L2TAG1 field of the Tx/Rx descriptors.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to insert hardware
+ * offloaded VLAN tags using the L2TAG2 field of the Tx descriptor.
+ *
+ * VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 - Used to tell the VF to strip hardware
+ * offloaded VLAN tags using the L2TAG2_2 field of the Rx descriptor.
+ *
+ * VIRTCHNL_VLAN_PRIO - This field supports VLAN priority bits. This is used for
+ * VLAN filtering if the underlying PF supports it.
+ *
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED - This field is used to say whether a
+ * certain VLAN capability can be toggled. For example if the underlying PF/CP
+ * allows the VF to toggle VLAN filtering, stripping, and/or insertion it should
+ * set this bit along with the supported ethertypes.
+ */
+enum virtchnl_vlan_support {
+ VIRTCHNL_VLAN_UNSUPPORTED = 0,
+ VIRTCHNL_VLAN_ETHERTYPE_8100 = 0x00000001,
+ VIRTCHNL_VLAN_ETHERTYPE_88A8 = 0x00000002,
+ VIRTCHNL_VLAN_ETHERTYPE_9100 = 0x00000004,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG1 = 0x00000100,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2 = 0x00000200,
+ VIRTCHNL_VLAN_TAG_LOCATION_L2TAG2_2 = 0x00000400,
+ VIRTCHNL_VLAN_PRIO = 0x01000000,
+ VIRTCHNL_VLAN_FILTER_MASK = 0x10000000,
+ VIRTCHNL_VLAN_ETHERTYPE_AND = 0x20000000,
+ VIRTCHNL_VLAN_ETHERTYPE_XOR = 0x40000000,
+ VIRTCHNL_VLAN_TOGGLE = 0x80000000
+};
+
+/* This structure is used as part of the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * for filtering, insertion, and stripping capabilities.
+ *
+ * If only outer capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective.
+ *
+ * If only inner capabilities are supported (for filtering, insertion, and/or
+ * stripping) then this refers to the outer most or single VLAN from the VF's
+ * perspective. Functionally this is the same as if only outer capabilities are
+ * supported. The VF driver is just forced to use the inner fields when
+ * adding/deleting filters and enabling/disabling offloads (if supported).
+ *
+ * If both outer and inner capabilities are supported (for filtering, insertion,
+ * and/or stripping) then outer refers to the outer most or single VLAN and
+ * inner refers to the second VLAN, if it exists, in the packet.
+ *
+ * There is no support for tunneled VLAN offloads, so outer or inner are never
+ * referring to a tunneled packet from the VF's perspective.
+ */
+struct virtchnl_vlan_supported_caps {
+ u32 outer;
+ u32 inner;
+};
+
+/* The PF populates these fields based on the supported VLAN filtering. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ADD_VLAN_V2 or VIRTCHNL_OP_DEL_VLAN_V2 messages using
+ * the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN filtering setting if the
+ * VIRTCHNL_VLAN_TOGGLE bit is set.
+ *
+ * The ethertype(s) specified in the ethertype_init field are the ethertypes
+ * enabled for VLAN filtering. VLAN filtering in this case refers to the outer
+ * most VLAN from the VF's perspective. If both inner and outer filtering are
+ * allowed then ethertype_init only refers to the outer most VLAN as only
+ * VLAN ethertype supported for inner VLAN filtering is
+ * VIRTCHNL_VLAN_ETHERTYPE_8100. By default, inner VLAN filtering is disabled
+ * when both inner and outer filtering are allowed.
+ *
+ * The max_filters field tells the VF how many VLAN filters it's allowed to have
+ * at any one time. If it exceeds this amount and tries to add another filter,
+ * then the request will be rejected by the PF. To prevent failures, the VF
+ * should keep track of how many VLAN filters it has added and not attempt to
+ * add more than max_filters.
+ */
+struct virtchnl_vlan_filtering_caps {
+ struct virtchnl_vlan_supported_caps filtering_support;
+ u32 ethertype_init;
+ u16 max_filters;
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_filtering_caps);
+
+/* This enum is used for the virtchnl_vlan_offload_caps structure to specify
+ * if the PF supports a different ethertype for stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION - The ethertype(s) specified
+ * for stripping affect the ethertype(s) specified for insertion and visa versa
+ * as well. If the VF tries to configure VLAN stripping via
+ * VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 with VIRTCHNL_VLAN_ETHERTYPE_8100 then
+ * that will be the ethertype for both stripping and insertion.
+ *
+ * VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED - The ethertype(s) specified for
+ * stripping do not affect the ethertype(s) specified for insertion and visa
+ * versa.
+ */
+enum virtchnl_vlan_ethertype_match {
+ VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION = 0,
+ VIRTCHNL_ETHERTYPE_MATCH_NOT_REQUIRED = 1,
+};
+
+/* The PF populates these fields based on the supported VLAN offloads. If a
+ * field is VIRTCHNL_VLAN_UNSUPPORTED then it's not supported and the PF will
+ * reject any VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 or
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 messages using the unsupported fields.
+ *
+ * Also, a VF is only allowed to toggle its VLAN offload setting if the
+ * VIRTCHNL_VLAN_TOGGLE_ALLOWED bit is set.
+ *
+ * The VF driver needs to be aware of how the tags are stripped by hardware and
+ * inserted by the VF driver based on the level of offload support. The PF will
+ * populate these fields based on where the VLAN tags are expected to be
+ * offloaded via the VIRTHCNL_VLAN_TAG_LOCATION_* bits. The VF will need to
+ * interpret these fields. See the definition of the
+ * VIRTCHNL_VLAN_TAG_LOCATION_* bits above the virtchnl_vlan_support
+ * enumeration.
+ */
+struct virtchnl_vlan_offload_caps {
+ struct virtchnl_vlan_supported_caps stripping_support;
+ struct virtchnl_vlan_supported_caps insertion_support;
+ u32 ethertype_init;
+ u8 ethertype_match;
+ u8 pad[3];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_vlan_offload_caps);
+
+/* VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS
+ * VF sends this message to determine its VLAN capabilities.
+ *
+ * PF will mark which capabilities it supports based on hardware support and
+ * current configuration. For example, if a port VLAN is configured the PF will
+ * not allow outer VLAN filtering, stripping, or insertion to be configured so
+ * it will block these features from the VF.
+ *
+ * The VF will need to cross reference its capabilities with the PFs
+ * capabilities in the response message from the PF to determine the VLAN
+ * support.
+ */
+struct virtchnl_vlan_caps {
+ struct virtchnl_vlan_filtering_caps filtering;
+ struct virtchnl_vlan_offload_caps offloads;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_caps);
+
+struct virtchnl_vlan {
+ u16 tci; /* tci[15:13] = PCP and tci[11:0] = VID */
+ u16 tci_mask; /* only valid if VIRTCHNL_VLAN_FILTER_MASK set in
+ * filtering caps
+ */
+ u16 tpid; /* 0x8100, 0x88a8, etc. and only type(s) set in
+ * filtering caps. Note that tpid here does not refer to
+ * VIRTCHNL_VLAN_ETHERTYPE_*, but it refers to the
+ * actual 2-byte VLAN TPID
+ */
+ u8 pad[2];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_vlan);
+
+struct virtchnl_vlan_filter {
+ struct virtchnl_vlan inner;
+ struct virtchnl_vlan outer;
+ u8 pad[16];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(32, virtchnl_vlan_filter);
+
+/* VIRTCHNL_OP_ADD_VLAN_V2
+ * VIRTCHNL_OP_DEL_VLAN_V2
+ *
+ * VF sends these messages to add/del one or more VLAN tag filters for Rx
+ * traffic.
+ *
+ * The PF attempts to add the filters and returns status.
+ *
+ * The VF should only ever attempt to add/del virtchnl_vlan_filter(s) using the
+ * supported fields negotiated via VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS.
+ */
+struct virtchnl_vlan_filter_list_v2 {
+ u16 vport_id;
+ u16 num_elements;
+ u8 pad[4];
+ struct virtchnl_vlan_filter filters[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_vlan_filter_list_v2);
+
+/* VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2
+ *
+ * VF sends this message to enable or disable VLAN stripping or insertion. It
+ * also needs to specify an ethertype. The VF knows which VLAN ethertypes are
+ * allowed and whether or not it's allowed to enable/disable the specific
+ * offload via the VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.offloads fields to determine which offload
+ * messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable and/or disable 0x8100 inner
+ * VLAN insertion and/or stripping via the opcodes listed above. Inner in this
+ * case means the outer most or single VLAN from the VF's perspective. This is
+ * because no outer offloads are supported. See the comments above the
+ * virtchnl_vlan_supported_caps structure for more details.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.inner =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * In order to enable inner (again note that in this case inner is the outer
+ * most or single VLAN from the VF's perspective) VLAN stripping for 0x8100
+ * VLANs, the VF would populate the virtchnl_vlan_setting structure in the
+ * following manner and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.inner_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * The reason that VLAN TPID(s) are not being used for the
+ * outer_ethertype_setting and inner_ethertype_setting fields is because it's
+ * possible a device could support VLAN insertion and/or stripping offload on
+ * multiple ethertypes concurrently, so this method allows a VF to request
+ * multiple ethertypes in one message using the virtchnl_vlan_support
+ * enumeration.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.offloads in the
+ * following manner the VF will be allowed to enable 0x8100 and 0x88a8 outer
+ * VLAN insertion and stripping simultaneously. The
+ * virtchnl_vlan_caps.offloads.ethertype_match field will also have to be
+ * populated based on what the PF can support.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN stripping for 0x8100 and 0x88a8 VLANs, the VF
+ * would populate the virthcnl_vlan_offload_structure in the following manner
+ * and send the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2 message.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTHCNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * There is also the case where a PF and the underlying hardware can support
+ * VLAN offloads on multiple ethertypes, but not concurrently. For example, if
+ * the PF populates the virtchnl_vlan_caps.offloads in the following manner the
+ * VF will be allowed to enable and/or disable 0x8100 XOR 0x88a8 outer VLAN
+ * offloads. The ethertypes must match for stripping and insertion.
+ *
+ * virtchnl_vlan_caps.offloads.stripping_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.insertion_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_XOR;
+ *
+ * virtchnl_vlan_caps.offloads.ethertype_match =
+ * VIRTCHNL_ETHERTYPE_STRIPPING_MATCHES_INSERTION;
+ *
+ * In order to enable outer VLAN stripping for 0x88a8 VLANs, the VF would
+ * populate the virtchnl_vlan_setting structure in the following manner and send
+ * the VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2. Also, this will change the
+ * ethertype for VLAN insertion if it's enabled. So, for completeness, a
+ * VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 with the same ethertype should be sent.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting = VIRTHCNL_VLAN_ETHERTYPE_88A8;
+ *
+ * virtchnl_vlan_setting.vport_id = vport_id or vsi_id assigned to the VF on
+ * initialization.
+ *
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2
+ *
+ * VF sends this message to enable or disable VLAN filtering. It also needs to
+ * specify an ethertype. The VF knows which VLAN ethertypes are allowed and
+ * whether or not it's allowed to enable/disable filtering via the
+ * VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS message. The VF needs to
+ * parse the virtchnl_vlan_caps.filtering fields to determine which, if any,
+ * filtering messages are allowed.
+ *
+ * For example, if the PF populates the virtchnl_vlan_caps.filtering in the
+ * following manner the VF will be allowed to enable/disable 0x8100 and 0x88a8
+ * outer VLAN filtering together. Note, that the VIRTCHNL_VLAN_ETHERTYPE_AND
+ * means that all filtering ethertypes will to be enabled and disabled together
+ * regardless of the request from the VF. This means that the underlying
+ * hardware only supports VLAN filtering for all VLAN the specified ethertypes
+ * or none of them.
+ *
+ * virtchnl_vlan_caps.filtering.filtering_support.outer =
+ * VIRTCHNL_VLAN_TOGGLE |
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTHCNL_VLAN_ETHERTYPE_88A8 |
+ * VIRTCHNL_VLAN_ETHERTYPE_9100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_AND;
+ *
+ * In order to enable outer VLAN filtering for 0x88a8 and 0x8100 VLANs (0x9100
+ * VLANs aren't supported by the VF driver), the VF would populate the
+ * virtchnl_vlan_setting structure in the following manner and send the
+ * VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2. The same message format would be used
+ * to disable outer VLAN filtering for 0x88a8 and 0x8100 VLANs, but the
+ * VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2 opcode is used.
+ *
+ * virtchnl_vlan_setting.outer_ethertype_setting =
+ * VIRTCHNL_VLAN_ETHERTYPE_8100 |
+ * VIRTCHNL_VLAN_ETHERTYPE_88A8;
+ *
+ */
+struct virtchnl_vlan_setting {
+ u32 outer_ethertype_setting;
+ u32 inner_ethertype_setting;
+ u16 vport_id;
+ u8 pad[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vlan_setting);
+
+/* VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE
+ * VF sends VSI id and flags.
+ * PF returns status code in retval.
+ * Note: we assume that broadcast accept mode is always enabled.
+ */
+struct virtchnl_promisc_info {
+ u16 vsi_id;
+ u16 flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_promisc_info);
+
+#define FLAG_VF_UNICAST_PROMISC 0x00000001
+#define FLAG_VF_MULTICAST_PROMISC 0x00000002
+
+/* VIRTCHNL_OP_GET_STATS
+ * VF sends this message to request stats for the selected VSI. VF uses
+ * the virtchnl_queue_select struct to specify the VSI. The queue_id
+ * field is ignored by the PF.
+ *
+ * PF replies with struct virtchnl_eth_stats in an external buffer.
+ */
+
+struct virtchnl_eth_stats {
+ u64 rx_bytes; /* received bytes */
+ u64 rx_unicast; /* received unicast pkts */
+ u64 rx_multicast; /* received multicast pkts */
+ u64 rx_broadcast; /* received broadcast pkts */
+ u64 rx_discards;
+ u64 rx_unknown_protocol;
+ u64 tx_bytes; /* transmitted bytes */
+ u64 tx_unicast; /* transmitted unicast pkts */
+ u64 tx_multicast; /* transmitted multicast pkts */
+ u64 tx_broadcast; /* transmitted broadcast pkts */
+ u64 tx_discards;
+ u64 tx_errors;
+};
+
+/* VIRTCHNL_OP_CONFIG_RSS_KEY
+ * VIRTCHNL_OP_CONFIG_RSS_LUT
+ * VF sends these messages to configure RSS. Only supported if both PF
+ * and VF drivers set the VIRTCHNL_VF_OFFLOAD_RSS_PF bit during
+ * configuration negotiation. If this is the case, then the RSS fields in
+ * the VF resource struct are valid.
+ * Both the key and LUT are initialized to 0 by the PF, meaning that
+ * RSS is effectively disabled until set up by the VF.
+ */
+struct virtchnl_rss_key {
+ u16 vsi_id;
+ u16 key_len;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_key);
+
+struct virtchnl_rss_lut {
+ u16 vsi_id;
+ u16 lut_entries;
+ u8 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(6, virtchnl_rss_lut);
+
+/* enum virthcnl_hash_filter
+ *
+ * Bits defining the hash filters in the hena field of the virtchnl_rss_hena
+ * structure. Each bit indicates a specific hash filter for RSS.
+ *
+ * Note that not all bits are supported on all hardware. The VF should use
+ * VIRTCHNL_OP_GET_RSS_HENA_CAPS to determine which bits the PF is capable of
+ * before using VIRTCHNL_OP_SET_RSS_HENA to enable specific filters.
+ */
+enum virtchnl_hash_filter {
+ /* Bits 0 through 28 are reserved for future use */
+ /* Bit 29, 30, and 32 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV4_UDP = 29,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV4_UDP = 30,
+ VIRTCHNL_HASH_FILTER_IPV4_UDP = 31,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP_SYN_NO_ACK = 32,
+ VIRTCHNL_HASH_FILTER_IPV4_TCP = 33,
+ VIRTCHNL_HASH_FILTER_IPV4_SCTP = 34,
+ VIRTCHNL_HASH_FILTER_IPV4_OTHER = 35,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV4 = 36,
+ /* Bits 37 and 38 are reserved for future use */
+ /* Bit 39, 40, and 42 are not supported on XL710 a X710 */
+ VIRTCHNL_HASH_FILTER_UNICAST_IPV6_UDP = 39,
+ VIRTCHNL_HASH_FILTER_MULTICAST_IPV6_UDP = 40,
+ VIRTCHNL_HASH_FILTER_IPV6_UDP = 41,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP_SYN_NO_ACK = 42,
+ VIRTCHNL_HASH_FILTER_IPV6_TCP = 43,
+ VIRTCHNL_HASH_FILTER_IPV6_SCTP = 44,
+ VIRTCHNL_HASH_FILTER_IPV6_OTHER = 45,
+ VIRTCHNL_HASH_FILTER_FRAG_IPV6 = 46,
+ /* Bit 37 is reserved for future use */
+ VIRTCHNL_HASH_FILTER_FCOE_OX = 48,
+ VIRTCHNL_HASH_FILTER_FCOE_RX = 49,
+ VIRTCHNL_HASH_FILTER_FCOE_OTHER = 50,
+ /* Bits 51 through 62 are reserved for future use */
+ VIRTCHNL_HASH_FILTER_L2_PAYLOAD = 63,
+};
+
+#define VIRTCHNL_HASH_FILTER_INVALID (0)
+
+/* VIRTCHNL_OP_GET_RSS_HENA_CAPS
+ * VIRTCHNL_OP_SET_RSS_HENA
+ * VF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the PF sets these to all possible traffic types that the
+ * hardware supports. The VF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ */
+struct virtchnl_rss_hena {
+ /* see enum virtchnl_hash_filter */
+ u64 hena;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_rss_hena);
+
+/* Type of RSS algorithm */
+enum virtchnl_rss_algorithm {
+ VIRTCHNL_RSS_ALG_TOEPLITZ_ASYMMETRIC = 0,
+ VIRTCHNL_RSS_ALG_R_ASYMMETRIC = 1,
+ VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC = 2,
+ VIRTCHNL_RSS_ALG_XOR_SYMMETRIC = 3,
+};
+
+/* This is used by PF driver to enforce how many channels can be supported.
+ * When ADQ_V2 capability is negotiated, it will allow 16 channels otherwise
+ * PF driver will allow only max 4 channels
+ */
+#define VIRTCHNL_MAX_ADQ_CHANNELS 4
+#define VIRTCHNL_MAX_ADQ_V2_CHANNELS 16
+
+/* VIRTCHNL_OP_ENABLE_CHANNELS
+ * VIRTCHNL_OP_DISABLE_CHANNELS
+ * VF sends these messages to enable or disable channels based on
+ * the user specified queue count and queue offset for each traffic class.
+ * This struct encompasses all the information that the PF needs from
+ * VF to create a channel.
+ */
+struct virtchnl_channel_info {
+ u16 count; /* number of queues in a channel */
+ u16 offset; /* queues in a channel start from 'offset' */
+ u32 pad;
+ u64 max_tx_rate;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_channel_info);
+
+struct virtchnl_tc_info {
+ u32 num_tc;
+ u32 pad;
+ struct virtchnl_channel_info list[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_tc_info);
+
+/* VIRTCHNL_ADD_CLOUD_FILTER
+ * VIRTCHNL_DEL_CLOUD_FILTER
+ * VF sends these messages to add or delete a cloud filter based on the
+ * user specified match and action filters. These structures encompass
+ * all the information that the PF needs from the VF to add/delete a
+ * cloud filter.
+ */
+
+struct virtchnl_l4_spec {
+ u8 src_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ u8 dst_mac[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ /* vlan_prio is part of this 16 bit field even from OS perspective
+ * vlan_id:12 is actual vlan_id, then vlanid:bit14..12 is vlan_prio
+ * in future, when decided to offload vlan_prio, pass that information
+ * as part of the "vlan_id" field, Bit14..12
+ */
+ __be16 vlan_id;
+ __be16 pad; /* reserved for future use */
+ __be32 src_ip[4];
+ __be32 dst_ip[4];
+ __be16 src_port;
+ __be16 dst_port;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(52, virtchnl_l4_spec);
+
+union virtchnl_flow_spec {
+ struct virtchnl_l4_spec tcp_spec;
+ u8 buffer[128]; /* reserved for future use */
+};
+
+VIRTCHNL_CHECK_UNION_LEN(128, virtchnl_flow_spec);
+
+enum virtchnl_action {
+ /* action types */
+ VIRTCHNL_ACTION_DROP = 0,
+ VIRTCHNL_ACTION_TC_REDIRECT,
+ VIRTCHNL_ACTION_PASSTHRU,
+ VIRTCHNL_ACTION_QUEUE,
+ VIRTCHNL_ACTION_Q_REGION,
+ VIRTCHNL_ACTION_MARK,
+ VIRTCHNL_ACTION_COUNT,
+};
+
+enum virtchnl_flow_type {
+ /* flow types */
+ VIRTCHNL_TCP_V4_FLOW = 0,
+ VIRTCHNL_TCP_V6_FLOW,
+ VIRTCHNL_UDP_V4_FLOW,
+ VIRTCHNL_UDP_V6_FLOW,
+};
+
+struct virtchnl_filter {
+ union virtchnl_flow_spec data;
+ union virtchnl_flow_spec mask;
+
+ /* see enum virtchnl_flow_type */
+ s32 flow_type;
+
+ /* see enum virtchnl_action */
+ s32 action;
+ u32 action_meta;
+ u8 field_flags;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(272, virtchnl_filter);
+
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+
+
+/* VIRTCHNL_OP_EVENT
+ * PF sends this message to inform the VF driver of events that may affect it.
+ * No direct response is expected from the VF, though it may generate other
+ * messages in response to this one.
+ */
+enum virtchnl_event_codes {
+ VIRTCHNL_EVENT_UNKNOWN = 0,
+ VIRTCHNL_EVENT_LINK_CHANGE,
+ VIRTCHNL_EVENT_RESET_IMPENDING,
+ VIRTCHNL_EVENT_PF_DRIVER_CLOSE,
+};
+
+#define PF_EVENT_SEVERITY_INFO 0
+#define PF_EVENT_SEVERITY_ATTENTION 1
+#define PF_EVENT_SEVERITY_ACTION_REQUIRED 2
+#define PF_EVENT_SEVERITY_CERTAIN_DOOM 255
+
+struct virtchnl_pf_event {
+ /* see enum virtchnl_event_codes */
+ s32 event;
+ union {
+ /* If the PF driver does not support the new speed reporting
+ * capabilities then use link_event else use link_event_adv to
+ * get the speed and link information. The ability to understand
+ * new speeds is indicated by setting the capability flag
+ * VIRTCHNL_VF_CAP_ADV_LINK_SPEED in vf_cap_flags parameter
+ * in virtchnl_vf_resource struct and can be used to determine
+ * which link event struct to use below.
+ */
+ struct {
+ enum virtchnl_link_speed link_speed;
+ bool link_status;
+ u8 pad[3];
+ } link_event;
+ struct {
+ /* link_speed provided in Mbps */
+ u32 link_speed;
+ u8 link_status;
+ u8 pad[3];
+ } link_event_adv;
+ } event_data;
+
+ s32 severity;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_pf_event);
+
+
+/* VF reset states - these are written into the RSTAT register:
+ * VFGEN_RSTAT on the VF
+ * When the PF initiates a reset, it writes 0
+ * When the reset is complete, it writes 1
+ * When the PF detects that the VF has recovered, it writes 2
+ * VF checks this register periodically to determine if a reset has occurred,
+ * then polls it to know when the reset is complete.
+ * If either the PF or VF reads the register while the hardware
+ * is in a reset state, it will return DEADBEEF, which, when masked
+ * will result in 3.
+ */
+enum virtchnl_vfr_states {
+ VIRTCHNL_VFR_INPROGRESS = 0,
+ VIRTCHNL_VFR_COMPLETED,
+ VIRTCHNL_VFR_VFACTIVE,
+};
+
+#define VIRTCHNL_MAX_NUM_PROTO_HDRS 32
+#define VIRTCHNL_MAX_SIZE_RAW_PACKET 1024
+#define PROTO_HDR_SHIFT 5
+#define PROTO_HDR_FIELD_START(proto_hdr_type) \
+ (proto_hdr_type << PROTO_HDR_SHIFT)
+#define PROTO_HDR_FIELD_MASK ((1UL << PROTO_HDR_SHIFT) - 1)
+
+/* VF use these macros to configure each protocol header.
+ * Specify which protocol headers and protocol header fields base on
+ * virtchnl_proto_hdr_type and virtchnl_proto_hdr_field.
+ * @param hdr: a struct of virtchnl_proto_hdr
+ * @param hdr_type: ETH/IPV4/TCP, etc
+ * @param field: SRC/DST/TEID/SPI, etc
+ */
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector |= BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, field) \
+ ((hdr)->field_selector &= ~BIT((field) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val) \
+ ((hdr)->field_selector & BIT((val) & PROTO_HDR_FIELD_MASK))
+#define VIRTCHNL_GET_PROTO_HDR_FIELD(hdr) ((hdr)->field_selector)
+
+#define VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_ADD_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+#define VIRTCHNL_DEL_PROTO_HDR_FIELD_BIT(hdr, hdr_type, field) \
+ (VIRTCHNL_DEL_PROTO_HDR_FIELD(hdr, \
+ VIRTCHNL_PROTO_HDR_ ## hdr_type ## _ ## field))
+
+#define VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, hdr_type) \
+ ((hdr)->type = VIRTCHNL_PROTO_HDR_ ## hdr_type)
+#define VIRTCHNL_GET_PROTO_HDR_TYPE(hdr) \
+ (((hdr)->type) >> PROTO_HDR_SHIFT)
+#define VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) \
+ ((hdr)->type == ((s32)((val) >> PROTO_HDR_SHIFT)))
+#define VIRTCHNL_TEST_PROTO_HDR(hdr, val) \
+ (VIRTCHNL_TEST_PROTO_HDR_TYPE(hdr, val) && \
+ VIRTCHNL_TEST_PROTO_HDR_FIELD(hdr, val))
+
+/* Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+enum virtchnl_proto_hdr_type {
+ VIRTCHNL_PROTO_HDR_NONE,
+ VIRTCHNL_PROTO_HDR_ETH,
+ VIRTCHNL_PROTO_HDR_S_VLAN,
+ VIRTCHNL_PROTO_HDR_C_VLAN,
+ VIRTCHNL_PROTO_HDR_IPV4,
+ VIRTCHNL_PROTO_HDR_IPV6,
+ VIRTCHNL_PROTO_HDR_TCP,
+ VIRTCHNL_PROTO_HDR_UDP,
+ VIRTCHNL_PROTO_HDR_SCTP,
+ VIRTCHNL_PROTO_HDR_GTPU_IP,
+ VIRTCHNL_PROTO_HDR_GTPU_EH,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN,
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP,
+ VIRTCHNL_PROTO_HDR_PPPOE,
+ VIRTCHNL_PROTO_HDR_L2TPV3,
+ VIRTCHNL_PROTO_HDR_ESP,
+ VIRTCHNL_PROTO_HDR_AH,
+ VIRTCHNL_PROTO_HDR_PFCP,
+ VIRTCHNL_PROTO_HDR_GTPC,
+ VIRTCHNL_PROTO_HDR_ECPRI,
+ VIRTCHNL_PROTO_HDR_L2TPV2,
+ VIRTCHNL_PROTO_HDR_PPP,
+ /* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL_PROTO_HDR_IPV4 and VIRTCHNL_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG,
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG,
+ VIRTCHNL_PROTO_HDR_GRE,
+};
+
+/* Protocol header field within a protocol header. */
+enum virtchnl_proto_hdr_field {
+ /* ETHER */
+ VIRTCHNL_PROTO_HDR_ETH_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ETH),
+ VIRTCHNL_PROTO_HDR_ETH_DST,
+ VIRTCHNL_PROTO_HDR_ETH_ETHERTYPE,
+ /* S-VLAN */
+ VIRTCHNL_PROTO_HDR_S_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_S_VLAN),
+ /* C-VLAN */
+ VIRTCHNL_PROTO_HDR_C_VLAN_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_C_VLAN),
+ /* IPV4 */
+ VIRTCHNL_PROTO_HDR_IPV4_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4),
+ VIRTCHNL_PROTO_HDR_IPV4_DST,
+ VIRTCHNL_PROTO_HDR_IPV4_DSCP,
+ VIRTCHNL_PROTO_HDR_IPV4_TTL,
+ VIRTCHNL_PROTO_HDR_IPV4_PROT,
+ VIRTCHNL_PROTO_HDR_IPV4_CHKSUM,
+ /* IPV6 */
+ VIRTCHNL_PROTO_HDR_IPV6_SRC =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6),
+ VIRTCHNL_PROTO_HDR_IPV6_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_TC,
+ VIRTCHNL_PROTO_HDR_IPV6_HOP_LIMIT,
+ VIRTCHNL_PROTO_HDR_IPV6_PROT,
+ /* IPV6 Prefix */
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX32_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX40_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX48_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX56_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX64_DST,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_SRC,
+ VIRTCHNL_PROTO_HDR_IPV6_PREFIX96_DST,
+ /* TCP */
+ VIRTCHNL_PROTO_HDR_TCP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_TCP),
+ VIRTCHNL_PROTO_HDR_TCP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_TCP_CHKSUM,
+ /* UDP */
+ VIRTCHNL_PROTO_HDR_UDP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_UDP),
+ VIRTCHNL_PROTO_HDR_UDP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_UDP_CHKSUM,
+ /* SCTP */
+ VIRTCHNL_PROTO_HDR_SCTP_SRC_PORT =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_SCTP),
+ VIRTCHNL_PROTO_HDR_SCTP_DST_PORT,
+ VIRTCHNL_PROTO_HDR_SCTP_CHKSUM,
+ /* GTPU_IP */
+ VIRTCHNL_PROTO_HDR_GTPU_IP_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_IP),
+ /* GTPU_EH */
+ VIRTCHNL_PROTO_HDR_GTPU_EH_PDU =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH),
+ VIRTCHNL_PROTO_HDR_GTPU_EH_QFI,
+ /* PPPOE */
+ VIRTCHNL_PROTO_HDR_PPPOE_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PPPOE),
+ /* L2TPV3 */
+ VIRTCHNL_PROTO_HDR_L2TPV3_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV3),
+ /* ESP */
+ VIRTCHNL_PROTO_HDR_ESP_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ESP),
+ /* AH */
+ VIRTCHNL_PROTO_HDR_AH_SPI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_AH),
+ /* PFCP */
+ VIRTCHNL_PROTO_HDR_PFCP_S_FIELD =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_PFCP),
+ VIRTCHNL_PROTO_HDR_PFCP_SEID,
+ /* GTPC */
+ VIRTCHNL_PROTO_HDR_GTPC_TEID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPC),
+ /* ECPRI */
+ VIRTCHNL_PROTO_HDR_ECPRI_MSG_TYPE =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_ECPRI),
+ VIRTCHNL_PROTO_HDR_ECPRI_PC_RTC_ID,
+ /* IPv4 Dummy Fragment */
+ VIRTCHNL_PROTO_HDR_IPV4_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV4_FRAG),
+ /* IPv6 Extension Fragment */
+ VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG_PKID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_IPV6_EH_FRAG),
+ /* GTPU_DWN/UP */
+ VIRTCHNL_PROTO_HDR_GTPU_DWN_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_DWN),
+ VIRTCHNL_PROTO_HDR_GTPU_UP_QFI =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_GTPU_EH_PDU_UP),
+ /* L2TPv2 */
+ VIRTCHNL_PROTO_HDR_L2TPV2_SESS_ID =
+ PROTO_HDR_FIELD_START(VIRTCHNL_PROTO_HDR_L2TPV2),
+ VIRTCHNL_PROTO_HDR_L2TPV2_LEN_SESS_ID,
+};
+
+struct virtchnl_proto_hdr {
+ /* see enum virtchnl_proto_hdr_type */
+ s32 type;
+ u32 field_selector; /* a bit mask to select field for header type */
+ u8 buffer[64];
+ /**
+ * binary buffer in network order for specific header type.
+ * For example, if type = VIRTCHNL_PROTO_HDR_IPV4, a IPv4
+ * header is expected to be copied into the buffer.
+ */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_proto_hdr);
+
+struct virtchnl_proto_hdrs {
+ u8 tunnel_level;
+ /**
+ * specify where protocol header start from.
+ * must be 0 when sending a raw packet request.
+ * 0 - from the outer layer
+ * 1 - from the first inner layer
+ * 2 - from the second inner layer
+ * ....
+ */
+ int count;
+ /**
+ * number of proto layers, must < VIRTCHNL_MAX_NUM_PROTO_HDRS
+ * must be 0 for a raw packet request.
+ */
+ union {
+ struct virtchnl_proto_hdr
+ proto_hdr[VIRTCHNL_MAX_NUM_PROTO_HDRS];
+ struct {
+ u16 pkt_len;
+ u8 spec[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ u8 mask[VIRTCHNL_MAX_SIZE_RAW_PACKET];
+ } raw;
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2312, virtchnl_proto_hdrs);
+
+struct virtchnl_rss_cfg {
+ struct virtchnl_proto_hdrs proto_hdrs; /* protocol headers */
+
+ /* see enum virtchnl_rss_algorithm; rss algorithm type */
+ s32 rss_algorithm;
+ u8 reserved[128]; /* reserve for future */
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2444, virtchnl_rss_cfg);
+
+/* action configuration for FDIR */
+struct virtchnl_filter_action {
+ /* see enum virtchnl_action type */
+ s32 type;
+ union {
+ /* used for queue and qgroup action */
+ struct {
+ u16 index;
+ u8 region;
+ } queue;
+ /* used for count action */
+ struct {
+ /* share counter ID with other flow rules */
+ u8 shared;
+ u32 id; /* counter ID */
+ } count;
+ /* used for mark action */
+ u32 mark_id;
+ u8 reserve[32];
+ } act_conf;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(36, virtchnl_filter_action);
+
+#define VIRTCHNL_MAX_NUM_ACTIONS 8
+
+struct virtchnl_filter_action_set {
+ /* action number must be less then VIRTCHNL_MAX_NUM_ACTIONS */
+ int count;
+ struct virtchnl_filter_action actions[VIRTCHNL_MAX_NUM_ACTIONS];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(292, virtchnl_filter_action_set);
+
+/* pattern and action for FDIR rule */
+struct virtchnl_fdir_rule {
+ struct virtchnl_proto_hdrs proto_hdrs;
+ struct virtchnl_filter_action_set action_set;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2604, virtchnl_fdir_rule);
+
+/* Status returned to VF after VF requests FDIR commands
+ * VIRTCHNL_FDIR_SUCCESS
+ * VF FDIR related request is successfully done by PF
+ * The request can be OP_ADD/DEL/QUERY_FDIR_FILTER.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE
+ * OP_ADD_FDIR_FILTER request is failed due to no Hardware resource.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_EXIST
+ * OP_ADD_FDIR_FILTER request is failed due to the rule is already existed.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT
+ * OP_ADD_FDIR_FILTER request is failed due to conflict with existing rule.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST
+ * OP_DEL_FDIR_FILTER request is failed due to this rule doesn't exist.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_INVALID
+ * OP_ADD_FDIR_FILTER request is failed due to parameters validation
+ * or HW doesn't support.
+ *
+ * VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT
+ * OP_ADD/DEL_FDIR_FILTER request is failed due to timing out
+ * for programming.
+ *
+ * VIRTCHNL_FDIR_FAILURE_QUERY_INVALID
+ * OP_QUERY_FDIR_FILTER request is failed due to parameters validation,
+ * for example, VF query counter of a rule who has no counter action.
+ */
+enum virtchnl_fdir_prgm_status {
+ VIRTCHNL_FDIR_SUCCESS = 0,
+ VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE,
+ VIRTCHNL_FDIR_FAILURE_RULE_EXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_CONFLICT,
+ VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST,
+ VIRTCHNL_FDIR_FAILURE_RULE_INVALID,
+ VIRTCHNL_FDIR_FAILURE_RULE_TIMEOUT,
+ VIRTCHNL_FDIR_FAILURE_QUERY_INVALID,
+};
+
+/* VIRTCHNL_OP_ADD_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id,
+ * validate_only and rule_cfg. PF will return flow_id
+ * if the request is successfully done and return add_status to VF.
+ */
+struct virtchnl_fdir_add {
+ u16 vsi_id; /* INPUT */
+ /*
+ * 1 for validating a fdir rule, 0 for creating a fdir rule.
+ * Validate and create share one ops: VIRTCHNL_OP_ADD_FDIR_FILTER.
+ */
+ u16 validate_only; /* INPUT */
+ u32 flow_id; /* OUTPUT */
+ struct virtchnl_fdir_rule rule_cfg; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(2616, virtchnl_fdir_add);
+
+/* VIRTCHNL_OP_DEL_FDIR_FILTER
+ * VF sends this request to PF by filling out vsi_id
+ * and flow_id. PF will return del_status to VF.
+ */
+struct virtchnl_fdir_del {
+ u16 vsi_id; /* INPUT */
+ u16 pad;
+ u32 flow_id; /* INPUT */
+
+ /* see enum virtchnl_fdir_prgm_status; OUTPUT */
+ s32 status;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(44, virtchnl_qos_cap_list);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP
+ * VF sends message virtchnl_queue_tc_mapping to set queue to tc
+ * mapping for all the Tx and Rx queues with a specified VSI, and
+ * would get response about bitmap of valid user priorities
+ * associated with queues.
+ */
+struct virtchnl_queue_tc_mapping {
+ u16 vsi_id;
+ u16 num_tc;
+ u16 num_queue_pairs;
+ u8 pad[2];
+ union {
+ struct {
+ u16 start_queue_id;
+ u16 queue_count;
+ } req;
+ struct {
+#define VIRTCHNL_USER_PRIO_TYPE_UP 0
+#define VIRTCHNL_USER_PRIO_TYPE_DSCP 1
+ u16 prio_type;
+ u16 valid_prio_bitmap;
+ } resp;
+ } tc[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_tc_mapping);
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queues_bw_cfg);
+
+/* queue types */
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl_queue_chunks {
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_chunks);
+
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ struct virtchnl_queue_chunks chunks;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_del_ena_dis_queues);
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_queue_vector_maps);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
+/* VIRTCHNL_VF_CAP_PTP
+ * VIRTCHNL_OP_1588_PTP_GET_CAPS
+ * VIRTCHNL_OP_1588_PTP_GET_TIME
+ * VIRTCHNL_OP_1588_PTP_SET_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ
+ * VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS
+ * VIRTCHNL_OP_1588_PTP_SET_PIN_CFG
+ * VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP
+ *
+ * Support for offloading control of the device PTP hardware clock (PHC) is enabled
+ * by VIRTCHNL_VF_CAP_PTP. This capability allows a VF to request that PF
+ * enable Tx and Rx timestamps, and request access to read and/or write the
+ * PHC on the device, as well as query if the VF has direct access to the PHC
+ * time registers.
+ *
+ * The VF must set VIRTCHNL_VF_CAP_PTP in its capabilities when requesting
+ * resources. If the capability is set in reply, the VF must then send
+ * a VIRTCHNL_OP_1588_PTP_GET_CAPS request during initialization. The VF indicates
+ * what extended capabilities it wants by setting the appropriate flags in the
+ * caps field. The PF reply will indicate what features are enabled for
+ * that VF.
+ */
+#define VIRTCHNL_1588_PTP_CAP_TX_TSTAMP BIT(0)
+#define VIRTCHNL_1588_PTP_CAP_RX_TSTAMP BIT(1)
+#define VIRTCHNL_1588_PTP_CAP_READ_PHC BIT(2)
+#define VIRTCHNL_1588_PTP_CAP_WRITE_PHC BIT(3)
+#define VIRTCHNL_1588_PTP_CAP_PHC_REGS BIT(4)
+#define VIRTCHNL_1588_PTP_CAP_PIN_CFG BIT(5)
+
+/**
+ * virtchnl_phc_regs
+ *
+ * Structure defines how the VF should access PHC related registers. The VF
+ * must request VIRTCHNL_1588_PTP_CAP_PHC_REGS. If the VF has access to PHC
+ * registers, the PF will reply with the capability flag set, and with this
+ * structure detailing what PCIe region and what offsets to use. If direct
+ * access is not available, this entire structure is reserved and the fields
+ * will be zero.
+ *
+ * If necessary in a future extension, a separate capability mutually
+ * exclusive with VIRTCHNL_1588_PTP_CAP_PHC_REGS might be used to change the
+ * entire format of this structure within virtchnl_ptp_caps.
+ *
+ * @clock_hi: Register offset of the high 32 bits of clock time
+ * @clock_lo: Register offset of the low 32 bits of clock time
+ * @pcie_region: The PCIe region the registers are located in.
+ * @rsvd: Reserved bits for future extension
+ */
+struct virtchnl_phc_regs {
+ u32 clock_hi;
+ u32 clock_lo;
+ u8 pcie_region;
+ u8 rsvd[15];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_regs);
+
+/* timestamp format enumeration
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_40BIT
+ *
+ * This format indicates a timestamp that uses the 40bit format from the
+ * flexible Rx descriptors. It is also the default Tx timestamp format used
+ * today.
+ *
+ * Such a timestamp has the following 40bit format:
+ *
+ * *--------------------------------*-------------------------------*-----------*
+ * | 32 bits of time in nanoseconds | 7 bits of sub-nanosecond time | valid bit |
+ * *--------------------------------*-------------------------------*-----------*
+ *
+ * The timestamp is passed in a u64, with the upper 24bits of the field
+ * reserved as zero.
+ *
+ * With this format, in order to report a full 64bit timestamp to userspace
+ * applications, the VF is responsible for performing timestamp extension by
+ * carefully comparing the timestamp with the PHC time. This can correctly
+ * be achieved with a recent cached copy of the PHC time by doing delta
+ * comparison between the 32bits of nanoseconds in the timestamp with the
+ * lower 32 bits of the clock time. For this to work, the cached PHC time
+ * must be from within 2^31 nanoseconds (~2.1 seconds) of when the timestamp
+ * was captured.
+ *
+ * VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS
+ *
+ * This format indicates a timestamp that is 64 bits of nanoseconds.
+ */
+enum virtchnl_ptp_tstamp_format {
+ VIRTCHNL_1588_PTP_TSTAMP_40BIT = 0,
+ VIRTCHNL_1588_PTP_TSTAMP_64BIT_NS = 1,
+};
+
+/**
+ * virtchnl_ptp_caps
+ *
+ * Structure that defines the PTP capabilities available to the VF. The VF
+ * sends VIRTCHNL_OP_1588_PTP_GET_CAPS, and must fill in the ptp_caps field
+ * indicating what capabilities it is requesting. The PF will respond with the
+ * same message with the virtchnl_ptp_caps structure indicating what is
+ * enabled for the VF.
+ *
+ * @phc_regs: If VIRTCHNL_1588_PTP_CAP_PHC_REGS is set, contains information
+ * on the PHC related registers available to the VF.
+ * @caps: On send, VF sets what capabilities it requests. On reply, PF
+ * indicates what has been enabled for this VF. The PF shall not set
+ * bits which were not requested by the VF.
+ * @max_adj: The maximum adjustment capable of being requested by
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ, in parts per billion. Note that 1 ppb
+ * is approximately 65.5 scaled_ppm. The PF shall clamp any
+ * frequency adjustment in VIRTCHNL_op_1588_ADJ_FREQ to +/- max_adj.
+ * Use of ppb in this field allows fitting the value into 4 bytes
+ * instead of potentially requiring 8 if scaled_ppm units were used.
+ * @tx_tstamp_idx: The Tx timestamp index to set in the transmit descriptor
+ * when requesting a timestamp for an outgoing packet.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is not enabled.
+ * @n_ext_ts: Number of external timestamp functions available. Reserved
+ * if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_per_out: Number of periodic output functions available. Reserved if
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @n_pins: Number of physical programmable pins able to be controlled.
+ * Reserved if VIRTCHNL_1588_PTP_CAP_PIN_CFG is not enabled.
+ * @tx_tstamp_format: Format of the Tx timestamps. Valid formats are defined
+ * by the virtchnl_ptp_tstamp enumeration. Note that Rx
+ * timestamps are tied to the descriptor format, and do not
+ * have a separate format field.
+ * @rsvd: Reserved bits for future extension.
+ *
+ * PTP capabilities
+ *
+ * VIRTCHNL_1588_PTP_CAP_TX_TSTAMP indicates that the VF can request transmit
+ * timestamps for packets in its transmit descriptors. If this is unset,
+ * transmit timestamp requests are ignored. Note that only one outstanding Tx
+ * timestamp request will be honored at a time. The PF shall handle receipt of
+ * the timestamp from the hardware, and will forward this to the VF by sending
+ * a VIRTCHNL_OP_1588_TX_TIMESTAMP message.
+ *
+ * VIRTCHNL_1588_PTP_CAP_RX_TSTAMP indicates that the VF receive queues have
+ * receive timestamps enabled in the flexible descriptors. Note that this
+ * requires a VF to also negotiate to enable advanced flexible descriptors in
+ * the receive path instead of the default legacy descriptor format.
+ *
+ * For a detailed description of the current Tx and Rx timestamp format, see
+ * the section on virtchnl_phc_tx_tstamp. Future extensions may indicate
+ * timestamp format in the capability structure.
+ *
+ * VIRTCHNL_1588_PTP_CAP_READ_PHC indicates that the VF may read the PHC time
+ * via the VIRTCHNL_OP_1588_PTP_GET_TIME command, or by directly reading PHC
+ * registers if VIRTCHNL_1588_PTP_CAP_PHC_REGS is also set.
+ *
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC indicates that the VF may request updates
+ * to the PHC time via VIRTCHNL_OP_1588_PTP_SET_TIME,
+ * VIRTCHNL_OP_1588_PTP_ADJ_TIME, and VIRTCHNL_OP_1588_PTP_ADJ_FREQ.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PHC_REGS indicates that the VF has direct access to
+ * certain PHC related registers, primarily for lower latency access to the
+ * PHC time. If this is set, the VF shall read the virtchnl_phc_regs section
+ * of the capabilities to determine the location of the clock registers. If
+ * this capability is not set, the entire 24 bytes of virtchnl_phc_regs is
+ * reserved as zero. Future extensions define alternative formats for this
+ * data, in which case they will be mutually exclusive with this capability.
+ *
+ * VIRTCHNL_1588_PTP_CAP_PIN_CFG indicates that the VF has the capability to
+ * control software defined pins. These pins can be assigned either as an
+ * input to timestamp external events, or as an output to cause a periodic
+ * signal output.
+ *
+ * Note that in the future, additional capability flags may be added which
+ * indicate additional extended support. All fields marked as reserved by this
+ * header will be set to zero. VF implementations should verify this to ensure
+ * that future extensions do not break compatibility.
+ */
+struct virtchnl_ptp_caps {
+ struct virtchnl_phc_regs phc_regs;
+ u32 caps;
+ s32 max_adj;
+ u8 tx_tstamp_idx;
+ u8 n_ext_ts;
+ u8 n_per_out;
+ u8 n_pins;
+ /* see enum virtchnl_ptp_tstamp_format */
+ u8 tx_tstamp_format;
+ u8 rsvd[11];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(48, virtchnl_ptp_caps);
+
+/**
+ * virtchnl_phc_time
+ * @time: PHC time in nanoseconds
+ * @rsvd: Reserved for future extension
+ *
+ * Structure sent with VIRTCHNL_OP_1588_PTP_SET_TIME and received with
+ * VIRTCHNL_OP_1588_PTP_GET_TIME. Contains the 64bits of PHC clock time in
+ * nanoseconds.
+ *
+ * VIRTCHNL_OP_1588_PTP_SET_TIME may be sent by the VF if
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set. This will request that the PHC time
+ * be set to the requested value. This operation is non-atomic and thus does
+ * not adjust for the delay between request and completion. It is recommended
+ * that the VF use VIRTCHNL_OP_1588_PTP_ADJ_TIME and
+ * VIRTCHNL_OP_1588_PTP_ADJ_FREQ when possible to steer the PHC clock.
+ *
+ * VIRTCHNL_OP_1588_PTP_GET_TIME may be sent to request the current time of
+ * the PHC. This op is available in case direct access via the PHC registers
+ * is not available.
+ */
+struct virtchnl_phc_time {
+ u64 time;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_time);
+
+/**
+ * virtchnl_phc_adj_time
+ * @delta: offset requested to adjust clock by
+ * @rsvd: reserved for future extension
+ *
+ * Sent with VIRTCHNL_OP_1588_PTP_ADJ_TIME. Used to request an adjustment of
+ * the clock time by the provided delta, with negative values representing
+ * subtraction. VIRTCHNL_OP_1588_PTP_ADJ_TIME may not be sent unless
+ * VIRTCHNL_1588_PTP_CAP_WRITE_PHC is set.
+ *
+ * The atomicity of this operation is not guaranteed. The PF should perform an
+ * atomic update using appropriate mechanisms if possible. However, this is
+ * not guaranteed.
+ */
+struct virtchnl_phc_adj_time {
+ s64 delta;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_time);
+
+/**
+ * virtchnl_phc_adj_freq
+ * @scaled_ppm: frequency adjustment represented in scaled parts per million
+ * @rsvd: Reserved for future extension
+ *
+ * Sent with the VIRTCHNL_OP_1588_PTP_ADJ_FREQ to request an adjustment to the
+ * clock frequency. The adjustment is in scaled_ppm, which is parts per
+ * million with a 16bit binary fractional portion. 1 part per billion is
+ * approximately 65.5 scaled_ppm.
+ *
+ * ppm = scaled_ppm / 2^16
+ *
+ * ppb = scaled_ppm * 1000 / 2^16 or
+ *
+ * ppb = scaled_ppm * 125 / 2^13
+ *
+ * The PF shall clamp any adjustment request to plus or minus the specified
+ * max_adj in the PTP capabilities.
+ *
+ * Requests for adjustment are always based off of nominal clock frequency and
+ * not compounding. To reset clock frequency, send a request with a scaled_ppm
+ * of 0.
+ */
+struct virtchnl_phc_adj_freq {
+ s64 scaled_ppm;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_adj_freq);
+
+/**
+ * virtchnl_phc_tx_stamp
+ * @tstamp: timestamp value
+ * @rsvd: Reserved for future extension
+ *
+ * Sent along with VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP from the PF when a Tx
+ * timestamp for the index associated with this VF in the tx_tstamp_idx field
+ * is captured by hardware.
+ *
+ * If VIRTCHNL_1588_PTP_CAP_TX_TSTAMP is set, the VF may request a timestamp
+ * for a packet in its transmit context descriptor by setting the appropriate
+ * flag and setting the timestamp index provided by the PF. On transmission,
+ * the timestamp will be captured and sent to the PF. The PF will forward this
+ * timestamp to the VF via the VIRTCHNL_1588_PTP_CAP_TX_TSTAMP op.
+ *
+ * The timestamp format is defined by the tx_tstamp_format field of the
+ * virtchnl_ptp_caps structure.
+ */
+struct virtchnl_phc_tx_tstamp {
+ u64 tstamp;
+ u8 rsvd[8];
+};
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_phc_tx_tstamp);
+
+enum virtchnl_phc_pin_func {
+ VIRTCHNL_PHC_PIN_FUNC_NONE = 0, /* Not assigned to any function */
+ VIRTCHNL_PHC_PIN_FUNC_EXT_TS = 1, /* Assigned to external timestamp */
+ VIRTCHNL_PHC_PIN_FUNC_PER_OUT = 2, /* Assigned to periodic output */
+};
+
+/* Length of the pin configuration data. All pin configurations belong within
+ * the same union and *must* have this length in bytes.
+ */
+#define VIRTCHNL_PIN_CFG_LEN 64
+
+/* virtchnl_phc_ext_ts_mode
+ *
+ * Mode of the external timestamp, indicating which edges of the input signal
+ * to timestamp.
+ */
+enum virtchnl_phc_ext_ts_mode {
+ VIRTCHNL_PHC_EXT_TS_NONE = 0,
+ VIRTCHNL_PHC_EXT_TS_RISING_EDGE = 1,
+ VIRTCHNL_PHC_EXT_TS_FALLING_EDGE = 2,
+ VIRTCHNL_PHC_EXT_TS_BOTH_EDGES = 3,
+};
+
+/**
+ * virtchnl_phc_ext_ts
+ * @mode: mode of external timestamp request
+ * @rsvd: reserved for future extension
+ *
+ * External timestamp configuration. Defines the configuration for this
+ * external timestamp function.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_NONE, the function is essentially disabled,
+ * timestamping nothing.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_RISING_EDGE, the function shall timestamp
+ * the rising edge of the input when it transitions from low to high signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_FALLING_EDGE, the function shall timestamp
+ * the falling edge of the input when it transitions from high to low signal.
+ *
+ * If mode is VIRTCHNL_PHC_EXT_TS_BOTH_EDGES, the function shall timestamp
+ * both the rising and falling edge of the signal whenever it changes.
+ *
+ * The PF shall return an error if the requested mode cannot be implemented on
+ * the function.
+ */
+struct virtchnl_phc_ext_ts {
+ u8 mode; /* see virtchnl_phc_ext_ts_mode */
+ u8 rsvd[63];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_ext_ts);
+
+/* virtchnl_phc_per_out_flags
+ *
+ * Flags defining periodic output functionality.
+ */
+enum virtchnl_phc_per_out_flags {
+ VIRTCHNL_PHC_PER_OUT_PHASE_START = BIT(0),
+};
+
+/**
+ * virtchnl_phc_per_out
+ * @start: absolute start time (if VIRTCHNL_PHC_PER_OUT_PHASE_START unset)
+ * @phase: phase offset to start (if VIRTCHNL_PHC_PER_OUT_PHASE_START set)
+ * @period: time to complete a full clock cycle (low - > high -> low)
+ * @on: length of time the signal should stay high
+ * @flags: flags defining the periodic output operation.
+ * rsvd: reserved for future extension
+ *
+ * Configuration for a periodic output signal. Used to define the signal that
+ * should be generated on a given function.
+ *
+ * The period field determines the full length of the clock cycle, including
+ * both duration hold high transition and duration to hold low transition in
+ * nanoseconds.
+ *
+ * The on field determines how long the signal should remain high. For
+ * a traditional square wave clock that is on for some duration and off for
+ * the same duration, use an on length of precisely half the period. The duty
+ * cycle of the clock is period/on.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is unset, then the request is to start
+ * a clock an absolute time. This means that the clock should start precisely
+ * at the specified time in the start field. If the start time is in the past,
+ * then the periodic output should start at the next valid multiple of the
+ * period plus the start time:
+ *
+ * new_start = (n * period) + start
+ * (choose n such that new start is in the future)
+ *
+ * Note that the PF should not reject a start time in the past because it is
+ * possible that such a start time was valid when the request was made, but
+ * became invalid due to delay in programming the pin.
+ *
+ * If VIRTCHNL_PHC_PER_OUT_PHASE_START is set, then the request is to start
+ * the next multiple of the period plus the phase offset. The phase must be
+ * less than the period. In this case, the clock should start as soon possible
+ * at the next available multiple of the period. To calculate a start time
+ * when programming this mode, use:
+ *
+ * start = (n * period) + phase
+ * (choose n such that start is in the future)
+ *
+ * A period of zero should be treated as a request to disable the clock
+ * output.
+ */
+struct virtchnl_phc_per_out {
+ union {
+ u64 start;
+ u64 phase;
+ };
+ u64 period;
+ u64 on;
+ u32 flags;
+ u8 rsvd[36];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(VIRTCHNL_PIN_CFG_LEN, virtchnl_phc_per_out);
+
+/* virtchnl_phc_pin_cfg_flags
+ *
+ * Definition of bits in the flags field of the virtchnl_phc_pin_cfg
+ * structure.
+ */
+enum virtchnl_phc_pin_cfg_flags {
+ /* Valid for VIRTCHNL_OP_1588_PTP_SET_PIN_CFG. If set, indicates this
+ * is a request to verify if the function can be assigned to the
+ * provided pin. In this case, the ext_ts and per_out fields are
+ * ignored, and the PF response must be an error if the pin cannot be
+ * assigned to that function index.
+ */
+ VIRTCHNL_PHC_PIN_CFG_VERIFY = BIT(0),
+};
+
+/**
+ * virtchnl_phc_set_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @ext_ts: external timestamp configuration
+ * @per_out: periodic output configuration
+ * @rsvd1: Reserved for future extension
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_SET_PIN_CFG op.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_SET_PIN_CFG to assign the pin to one
+ * of the functions. It must set the pin_index field, the func field, and
+ * the func_index field. The pin_index must be less than n_pins, and the
+ * func_index must be less than the n_ext_ts or n_per_out depending on which
+ * function type is selected. If func is for an external timestamp, the
+ * ext_ts field must be filled in with the desired configuration. Similarly,
+ * if the function is for a periodic output, the per_out field must be
+ * configured.
+ *
+ * If the VIRTCHNL_PHC_PIN_CFG_VERIFY bit of the flag field is set, this is
+ * a request only to verify the configuration, not to set it. In this case,
+ * the PF should simply report an error if the requested pin cannot be
+ * assigned to the requested function. This allows VF to determine whether or
+ * not a given function can be assigned to a specific pin. Other flag bits are
+ * currently reserved and must be verified as zero on both sides. They may be
+ * extended in the future.
+ */
+struct virtchnl_phc_set_pin {
+ u32 flags; /* see virtchnl_phc_pin_cfg_flags */
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd1;
+ union {
+ struct virtchnl_phc_ext_ts ext_ts;
+ struct virtchnl_phc_per_out per_out;
+ };
+ u8 rsvd2[8];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_set_pin);
+
+/**
+ * virtchnl_phc_pin
+ * @pin_index: The pin to get or set
+ * @func: the function type the pin is assigned to
+ * @func_index: the index of the function the pin is assigned to
+ * @rsvd: Reserved for future extension
+ * @name: human readable pin name, supplied by PF on GET_PIN_CFGS
+ *
+ * Sent by the PF as part of the VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS response.
+ *
+ * The VF issues a VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS request to the PF in
+ * order to obtain the current pin configuration for all of the pins that were
+ * assigned to this VF.
+ *
+ * This structure details the pin configuration state, including a pin name
+ * and which function is assigned to the pin currently.
+ */
+struct virtchnl_phc_pin {
+ u8 pin_index;
+ u8 func; /* see virtchnl_phc_pin_func */
+ u8 func_index;
+ u8 rsvd[5];
+ char name[64];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(72, virtchnl_phc_pin);
+
+/**
+ * virtchnl_phc_pin_cfg
+ * @len: length of the variable pin config array
+ * @pins: variable length pin configuration array
+ *
+ * Variable structure sent by the PF in reply to
+ * VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS. The VF does not send this structure with
+ * its request of the operation.
+ *
+ * It is possible that the PF may need to send more pin configuration data
+ * than can be sent in one virtchnl message. To handle this, the PF should
+ * issue multiple VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS responses. Each response
+ * will indicate the number of pins it covers. The VF should be ready to wait
+ * for multiple responses until it has received a total length equal to the
+ * number of n_pins negotiated during extended PTP capabilities exchange.
+ */
+struct virtchnl_phc_get_pins {
+ u8 len;
+ u8 rsvd[7];
+ struct virtchnl_phc_pin pins[1];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(80, virtchnl_phc_get_pins);
+
+/**
+ * virtchnl_phc_ext_stamp
+ * @tstamp: timestamp value
+ * @tstamp_rsvd: Reserved for future extension of the timestamp value.
+ * @tstamp_format: format of the timstamp
+ * @func_index: external timestamp function this timestamp is for
+ * @rsvd2: Reserved for future extension
+ *
+ * Sent along with the VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP from the PF when an
+ * external timestamp function is triggered.
+ *
+ * This will be sent only if one of the external timestamp functions is
+ * configured by the VF, and is only valid if VIRTCHNL_1588_PTP_CAP_PIN_CFG is
+ * negotiated with the PF.
+ *
+ * The timestamp format is defined by the tstamp_format field using the
+ * virtchnl_ptp_tstamp_format enumeration. The tstamp_rsvd field is
+ * exclusively reserved for possible future variants of the timestamp format,
+ * and its access will be controlled by the tstamp_format field.
+ */
+struct virtchnl_phc_ext_tstamp {
+ u64 tstamp;
+ u8 tstamp_rsvd[8];
+ u8 tstamp_format;
+ u8 func_index;
+ u8 rsvd2[6];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(24, virtchnl_phc_ext_tstamp);
+
+/* Since VF messages are limited by u16 size, precalculate the maximum possible
+ * values of nested elements in virtchnl structures that virtual channel can
+ * possibly handle in a single message.
+ */
+enum virtchnl_vector_limits {
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vsi_queue_config_info)) /
+ sizeof(struct virtchnl_queue_pair_info),
+
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_irq_map_info)) /
+ sizeof(struct virtchnl_vector_map),
+
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_ether_addr_list)) /
+ sizeof(struct virtchnl_ether_addr),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list)) /
+ sizeof(u16),
+
+
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_tc_info)) /
+ sizeof(struct virtchnl_channel_info),
+
+ VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_del_ena_dis_queues)) /
+ sizeof(struct virtchnl_queue_chunk),
+
+ VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_queue_vector_maps)) /
+ sizeof(struct virtchnl_queue_vector),
+
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX =
+ ((u16)(~0) - sizeof(struct virtchnl_vlan_filter_list_v2)) /
+ sizeof(struct virtchnl_vlan_filter),
+};
+
+/**
+ * virtchnl_vc_validate_vf_msg
+ * @ver: Virtchnl version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
+ u8 *msg, u16 msglen)
+{
+ bool err_msg_format = false;
+ u32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL_OP_VERSION:
+ valid_len = sizeof(struct virtchnl_version_info);
+ break;
+ case VIRTCHNL_OP_RESET_VF:
+ break;
+ case VIRTCHNL_OP_GET_VF_RESOURCES:
+ if (VF_IS_V11(ver))
+ valid_len = sizeof(u32);
+ break;
+ case VIRTCHNL_OP_CONFIG_TX_QUEUE:
+ valid_len = sizeof(struct virtchnl_txq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_RX_QUEUE:
+ valid_len = sizeof(struct virtchnl_rxq_info);
+ break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ valid_len = sizeof(struct virtchnl_vsi_queue_config_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_vsi_queue_config_info *vqc =
+ (struct virtchnl_vsi_queue_config_info *)msg;
+
+ if (vqc->num_queue_pairs == 0 || vqc->num_queue_pairs >
+ VIRTCHNL_OP_CONFIG_VSI_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vqc->num_queue_pairs *
+ sizeof(struct
+ virtchnl_queue_pair_info));
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ valid_len = sizeof(struct virtchnl_irq_map_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_irq_map_info *vimi =
+ (struct virtchnl_irq_map_info *)msg;
+
+ if (vimi->num_vectors == 0 || vimi->num_vectors >
+ VIRTCHNL_OP_CONFIG_IRQ_MAP_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vimi->num_vectors *
+ sizeof(struct virtchnl_vector_map));
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
+ case VIRTCHNL_OP_ADD_ETH_ADDR:
+ case VIRTCHNL_OP_DEL_ETH_ADDR:
+ valid_len = sizeof(struct virtchnl_ether_addr_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_ether_addr_list *veal =
+ (struct virtchnl_ether_addr_list *)msg;
+
+ if (veal->num_elements == 0 || veal->num_elements >
+ VIRTCHNL_OP_ADD_DEL_ETH_ADDR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += veal->num_elements *
+ sizeof(struct virtchnl_ether_addr);
+ }
+ break;
+ case VIRTCHNL_OP_ADD_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list *vfl =
+ (struct virtchnl_vlan_filter_list *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += vfl->num_elements * sizeof(u16);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE:
+ valid_len = sizeof(struct virtchnl_promisc_info);
+ break;
+ case VIRTCHNL_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl_queue_select);
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_KEY:
+ valid_len = sizeof(struct virtchnl_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_key *vrk =
+ (struct virtchnl_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_RSS_LUT:
+ valid_len = sizeof(struct virtchnl_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl_rss_lut *vrl =
+ (struct virtchnl_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += vrl->lut_entries - 1;
+ }
+ break;
+ case VIRTCHNL_OP_GET_RSS_HENA_CAPS:
+ break;
+ case VIRTCHNL_OP_SET_RSS_HENA:
+ valid_len = sizeof(struct virtchnl_rss_hena);
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING:
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ valid_len = sizeof(struct virtchnl_vf_res_request);
+ break;
+ case VIRTCHNL_OP_ENABLE_CHANNELS:
+ valid_len = sizeof(struct virtchnl_tc_info);
+ if (msglen >= valid_len) {
+ struct virtchnl_tc_info *vti =
+ (struct virtchnl_tc_info *)msg;
+
+ if (vti->num_tc == 0 || vti->num_tc >
+ VIRTCHNL_OP_ENABLE_CHANNELS_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vti->num_tc - 1) *
+ sizeof(struct virtchnl_channel_info);
+ }
+ break;
+ case VIRTCHNL_OP_DISABLE_CHANNELS:
+ break;
+ case VIRTCHNL_OP_ADD_CLOUD_FILTER:
+ case VIRTCHNL_OP_DEL_CLOUD_FILTER:
+ valid_len = sizeof(struct virtchnl_filter);
+ break;
+ case VIRTCHNL_OP_ADD_RSS_CFG:
+ case VIRTCHNL_OP_DEL_RSS_CFG:
+ valid_len = sizeof(struct virtchnl_rss_cfg);
+ break;
+ case VIRTCHNL_OP_ADD_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_add);
+ break;
+ case VIRTCHNL_OP_DEL_FDIR_FILTER:
+ valid_len = sizeof(struct virtchnl_fdir_del);
+ break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_TC_MAP:
+ valid_len = sizeof(struct virtchnl_queue_tc_mapping);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_tc_mapping *q_tc =
+ (struct virtchnl_queue_tc_mapping *)msg;
+ if (q_tc->num_tc == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_tc->num_tc - 1) *
+ sizeof(q_tc->tc[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = sizeof(struct virtchnl_queues_bw_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (q_bw->num_queues - 1) *
+ sizeof(q_bw->cfg[0]);
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS:
+ break;
+ case VIRTCHNL_OP_ADD_VLAN_V2:
+ case VIRTCHNL_OP_DEL_VLAN_V2:
+ valid_len = sizeof(struct virtchnl_vlan_filter_list_v2);
+ if (msglen >= valid_len) {
+ struct virtchnl_vlan_filter_list_v2 *vfl =
+ (struct virtchnl_vlan_filter_list_v2 *)msg;
+
+ if (vfl->num_elements == 0 || vfl->num_elements >
+ VIRTCHNL_OP_ADD_DEL_VLAN_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+
+ valid_len += (vfl->num_elements - 1) *
+ sizeof(struct virtchnl_vlan_filter);
+ }
+ break;
+ case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
+ case VIRTCHNL_OP_ENABLE_VLAN_FILTERING_V2:
+ case VIRTCHNL_OP_DISABLE_VLAN_FILTERING_V2:
+ valid_len = sizeof(struct virtchnl_vlan_setting);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl_ptp_caps);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_TIME:
+ case VIRTCHNL_OP_1588_PTP_SET_TIME:
+ valid_len = sizeof(struct virtchnl_phc_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_TIME:
+ valid_len = sizeof(struct virtchnl_phc_adj_time);
+ break;
+ case VIRTCHNL_OP_1588_PTP_ADJ_FREQ:
+ valid_len = sizeof(struct virtchnl_phc_adj_freq);
+ break;
+ case VIRTCHNL_OP_1588_PTP_TX_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_tx_tstamp);
+ break;
+ case VIRTCHNL_OP_1588_PTP_SET_PIN_CFG:
+ valid_len = sizeof(struct virtchnl_phc_set_pin);
+ break;
+ case VIRTCHNL_OP_1588_PTP_GET_PIN_CFGS:
+ break;
+ case VIRTCHNL_OP_1588_PTP_EXT_TIMESTAMP:
+ valid_len = sizeof(struct virtchnl_phc_ext_tstamp);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL_OP_ENABLE_DISABLE_DEL_QUEUES_V2_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl_queue_chunk);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp =
+ (struct virtchnl_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl_queue_vector);
+ }
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_EVENT:
+ case VIRTCHNL_OP_UNKNOWN:
+ default:
+ return VIRTCHNL_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+#endif /* _VIRTCHNL_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2.h b/drivers/common/idpf/base/virtchnl2.h
new file mode 100644
index 0000000000..d496f2388e
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2.h
@@ -0,0 +1,1462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL2_H_
+#define _VIRTCHNL2_H_
+
+/* All opcodes associated with virtchnl 2 are prefixed with virtchnl2 or
+ * VIRTCHNL2. Any future opcodes, offloads/capabilities, structures,
+ * and defines must be prefixed with virtchnl2 or VIRTCHNL2 to avoid confusion.
+ */
+
+#include "virtchnl2_lan_desc.h"
+
+/* Error Codes
+ * Note that many older versions of various iAVF drivers convert the reported
+ * status code directly into an iavf_status enumeration. For this reason, it
+ * is important that the values of these enumerations line up.
+ */
+#define VIRTCHNL2_STATUS_SUCCESS 0
+#define VIRTCHNL2_STATUS_ERR_PARAM -5
+#define VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH -38
+
+/* These macros are used to generate compilation errors if a structure/union
+ * is not exactly the correct length. It gives a divide by zero error if the
+ * structure/union is not of the correct size, otherwise it creates an enum
+ * that is never used.
+ */
+#define VIRTCHNL2_CHECK_STRUCT_LEN(n, X) enum virtchnl2_static_assert_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(struct X) == (n)) ? 1 : 0) }
+#define VIRTCHNL2_CHECK_UNION_LEN(n, X) enum virtchnl2_static_asset_enum_##X \
+ { virtchnl2_static_assert_##X = (n)/((sizeof(union X) == (n)) ? 1 : 0) }
+
+/* New major set of opcodes introduced and so leaving room for
+ * old misc opcodes to be added in future. Also these opcodes may only
+ * be used if both the PF and VF have successfully negotiated the
+ * VIRTCHNL version as 2.0 during VIRTCHNL22_OP_VERSION exchange.
+ */
+#define VIRTCHNL2_OP_UNKNOWN 0
+#define VIRTCHNL2_OP_VERSION 1
+#define VIRTCHNL2_OP_GET_CAPS 500
+#define VIRTCHNL2_OP_CREATE_VPORT 501
+#define VIRTCHNL2_OP_DESTROY_VPORT 502
+#define VIRTCHNL2_OP_ENABLE_VPORT 503
+#define VIRTCHNL2_OP_DISABLE_VPORT 504
+#define VIRTCHNL2_OP_CONFIG_TX_QUEUES 505
+#define VIRTCHNL2_OP_CONFIG_RX_QUEUES 506
+#define VIRTCHNL2_OP_ENABLE_QUEUES 507
+#define VIRTCHNL2_OP_DISABLE_QUEUES 508
+#define VIRTCHNL2_OP_ADD_QUEUES 509
+#define VIRTCHNL2_OP_DEL_QUEUES 510
+#define VIRTCHNL2_OP_MAP_QUEUE_VECTOR 511
+#define VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR 512
+#define VIRTCHNL2_OP_GET_RSS_KEY 513
+#define VIRTCHNL2_OP_SET_RSS_KEY 514
+#define VIRTCHNL2_OP_GET_RSS_LUT 515
+#define VIRTCHNL2_OP_SET_RSS_LUT 516
+#define VIRTCHNL2_OP_GET_RSS_HASH 517
+#define VIRTCHNL2_OP_SET_RSS_HASH 518
+#define VIRTCHNL2_OP_SET_SRIOV_VFS 519
+#define VIRTCHNL2_OP_ALLOC_VECTORS 520
+#define VIRTCHNL2_OP_DEALLOC_VECTORS 521
+#define VIRTCHNL2_OP_EVENT 522
+#define VIRTCHNL2_OP_GET_STATS 523
+#define VIRTCHNL2_OP_RESET_VF 524
+ /* opcode 525 is reserved */
+#define VIRTCHNL2_OP_GET_PTYPE_INFO 526
+ /* opcode 527 and 528 are reserved for VIRTCHNL2_OP_GET_PTYPE_ID and
+ * VIRTCHNL2_OP_GET_PTYPE_INFO_RAW
+ */
+ /* opcodes 529, 530, and 531 are reserved */
+#define VIRTCHNL2_OP_CREATE_ADI 532
+#define VIRTCHNL2_OP_DESTROY_ADI 533
+#define VIRTCHNL2_OP_LOOPBACK 534
+#define VIRTCHNL2_OP_ADD_MAC_ADDR 535
+#define VIRTCHNL2_OP_DEL_MAC_ADDR 536
+#define VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE 537
+
+#define VIRTCHNL2_RDMA_INVALID_QUEUE_IDX 0xFFFF
+
+/* VIRTCHNL2_VPORT_TYPE
+ * Type of virtual port
+ */
+#define VIRTCHNL2_VPORT_TYPE_DEFAULT 0
+#define VIRTCHNL2_VPORT_TYPE_SRIOV 1
+#define VIRTCHNL2_VPORT_TYPE_SIOV 2
+#define VIRTCHNL2_VPORT_TYPE_SUBDEV 3
+#define VIRTCHNL2_VPORT_TYPE_MNG 4
+
+/* VIRTCHNL2_QUEUE_MODEL
+ * Type of queue model
+ *
+ * In the single queue model, the same transmit descriptor queue is used by
+ * software to post descriptors to hardware and by hardware to post completed
+ * descriptors to software.
+ * Likewise, the same receive descriptor queue is used by hardware to post
+ * completions to software and by software to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SINGLE 0
+/* In the split queue model, hardware uses transmit completion queues to post
+ * descriptor/buffer completions to software, while software uses transmit
+ * descriptor queues to post descriptors to hardware.
+ * Likewise, hardware posts descriptor completions to the receive descriptor
+ * queue, while software uses receive buffer queues to post buffers to hardware.
+ */
+#define VIRTCHNL2_QUEUE_MODEL_SPLIT 1
+
+/* VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS
+ * Checksum offload capability flags
+ */
+#define VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 BIT(0)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP BIT(1)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP BIT(2)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP BIT(3)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_TX_CSUM_GENERIC BIT(7)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 BIT(8)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP BIT(9)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP BIT(10)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP BIT(11)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP BIT(12)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP BIT(13)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP BIT(14)
+#define VIRTCHNL2_CAP_RX_CSUM_GENERIC BIT(15)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_SINGLE_TUNNEL BIT(16)
+#define VIRTCHNL2_CAP_TX_CSUM_L3_DOUBLE_TUNNEL BIT(17)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_SINGLE_TUNNEL BIT(18)
+#define VIRTCHNL2_CAP_RX_CSUM_L3_DOUBLE_TUNNEL BIT(19)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_SINGLE_TUNNEL BIT(20)
+#define VIRTCHNL2_CAP_TX_CSUM_L4_DOUBLE_TUNNEL BIT(21)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_SINGLE_TUNNEL BIT(22)
+#define VIRTCHNL2_CAP_RX_CSUM_L4_DOUBLE_TUNNEL BIT(23)
+
+/* VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS
+ * Segmentation offload capability flags
+ */
+#define VIRTCHNL2_CAP_SEG_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_SEG_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_SEG_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_SEG_IPV6_TCP BIT(3)
+#define VIRTCHNL2_CAP_SEG_IPV6_UDP BIT(4)
+#define VIRTCHNL2_CAP_SEG_IPV6_SCTP BIT(5)
+#define VIRTCHNL2_CAP_SEG_GENERIC BIT(6)
+#define VIRTCHNL2_CAP_SEG_TX_SINGLE_TUNNEL BIT(7)
+#define VIRTCHNL2_CAP_SEG_TX_DOUBLE_TUNNEL BIT(8)
+
+/* VIRTCHNL2_RSS_FLOW_TYPE_CAPS
+ * Receive Side Scaling Flow type capability flags
+ */
+#define VIRTCHNL2_CAP_RSS_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSS_IPV4_UDP BIT(1)
+#define VIRTCHNL2_CAP_RSS_IPV4_SCTP BIT(2)
+#define VIRTCHNL2_CAP_RSS_IPV4_OTHER BIT(3)
+#define VIRTCHNL2_CAP_RSS_IPV6_TCP BIT(4)
+#define VIRTCHNL2_CAP_RSS_IPV6_UDP BIT(5)
+#define VIRTCHNL2_CAP_RSS_IPV6_SCTP BIT(6)
+#define VIRTCHNL2_CAP_RSS_IPV6_OTHER BIT(7)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH BIT(8)
+#define VIRTCHNL2_CAP_RSS_IPV4_ESP BIT(9)
+#define VIRTCHNL2_CAP_RSS_IPV4_AH_ESP BIT(10)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH BIT(11)
+#define VIRTCHNL2_CAP_RSS_IPV6_ESP BIT(12)
+#define VIRTCHNL2_CAP_RSS_IPV6_AH_ESP BIT(13)
+
+/* VIRTCHNL2_HEADER_SPLIT_CAPS
+ * Header split capability flags
+ */
+/* for prepended metadata */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 BIT(0)
+/* all VLANs go into header buffer */
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 BIT(1)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 BIT(2)
+#define VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6 BIT(3)
+
+/* VIRTCHNL2_RSC_OFFLOAD_CAPS
+ * Receive Side Coalescing offload capability flags
+ */
+#define VIRTCHNL2_CAP_RSC_IPV4_TCP BIT(0)
+#define VIRTCHNL2_CAP_RSC_IPV4_SCTP BIT(1)
+#define VIRTCHNL2_CAP_RSC_IPV6_TCP BIT(2)
+#define VIRTCHNL2_CAP_RSC_IPV6_SCTP BIT(3)
+
+/* VIRTCHNL2_OTHER_CAPS
+ * Other capability flags
+ * SPLITQ_QSCHED: Queue based scheduling using split queue model
+ * TX_VLAN: VLAN tag insertion
+ * RX_VLAN: VLAN tag stripping
+ */
+#define VIRTCHNL2_CAP_RDMA BIT(0)
+#define VIRTCHNL2_CAP_SRIOV BIT(1)
+#define VIRTCHNL2_CAP_MACFILTER BIT(2)
+#define VIRTCHNL2_CAP_FLOW_DIRECTOR BIT(3)
+#define VIRTCHNL2_CAP_SPLITQ_QSCHED BIT(4)
+#define VIRTCHNL2_CAP_CRC BIT(5)
+#define VIRTCHNL2_CAP_ADQ BIT(6)
+#define VIRTCHNL2_CAP_WB_ON_ITR BIT(7)
+#define VIRTCHNL2_CAP_PROMISC BIT(8)
+#define VIRTCHNL2_CAP_LINK_SPEED BIT(9)
+#define VIRTCHNL2_CAP_INLINE_IPSEC BIT(10)
+#define VIRTCHNL2_CAP_LARGE_NUM_QUEUES BIT(11)
+/* require additional info */
+#define VIRTCHNL2_CAP_VLAN BIT(12)
+#define VIRTCHNL2_CAP_PTP BIT(13)
+#define VIRTCHNL2_CAP_ADV_RSS BIT(15)
+#define VIRTCHNL2_CAP_FDIR BIT(16)
+#define VIRTCHNL2_CAP_RX_FLEX_DESC BIT(17)
+#define VIRTCHNL2_CAP_PTYPE BIT(18)
+#define VIRTCHNL2_CAP_LOOPBACK BIT(19)
+#define VIRTCHNL2_CAP_OEM BIT(20)
+
+/* VIRTCHNL2_DEVICE_TYPE */
+/* underlying device type */
+#define VIRTCHNL2_MEV_DEVICE 0
+
+/* VIRTCHNL2_TXQ_SCHED_MODE
+ * Transmit Queue Scheduling Modes - Queue mode is the legacy mode i.e. inorder
+ * completions where descriptors and buffers are completed at the same time.
+ * Flow scheduling mode allows for out of order packet processing where
+ * descriptors are cleaned in order, but buffers can be completed out of order.
+ */
+#define VIRTCHNL2_TXQ_SCHED_MODE_QUEUE 0
+#define VIRTCHNL2_TXQ_SCHED_MODE_FLOW 1
+
+/* VIRTCHNL2_TXQ_FLAGS
+ * Transmit Queue feature flags
+ *
+ * Enable rule miss completion type; packet completion for a packet
+ * sent on exception path; only relevant in flow scheduling mode
+ */
+#define VIRTCHNL2_TXQ_ENABLE_MISS_COMPL BIT(0)
+
+/* VIRTCHNL2_PEER_TYPE
+ * Transmit mailbox peer type
+ */
+#define VIRTCHNL2_RDMA_CPF 0
+#define VIRTCHNL2_NVME_CPF 1
+#define VIRTCHNL2_ATE_CPF 2
+#define VIRTCHNL2_LCE_CPF 3
+
+/* VIRTCHNL2_RXQ_FLAGS
+ * Receive Queue Feature flags
+ */
+#define VIRTCHNL2_RXQ_RSC BIT(0)
+#define VIRTCHNL2_RXQ_HDR_SPLIT BIT(1)
+/* When set, packet descriptors are flushed by hardware immediately after
+ * processing each packet.
+ */
+#define VIRTCHNL2_RXQ_IMMEDIATE_WRITE_BACK BIT(2)
+#define VIRTCHNL2_RX_DESC_SIZE_16BYTE BIT(3)
+#define VIRTCHNL2_RX_DESC_SIZE_32BYTE BIT(4)
+
+/* VIRTCHNL2_RSS_ALGORITHM
+ * Type of RSS algorithm
+ */
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_ASYMMETRIC 0
+#define VIRTCHNL2_RSS_ALG_R_ASYMMETRIC 1
+#define VIRTCHNL2_RSS_ALG_TOEPLITZ_SYMMETRIC 2
+#define VIRTCHNL2_RSS_ALG_XOR_SYMMETRIC 3
+
+/* VIRTCHNL2_EVENT_CODES
+ * Type of event
+ */
+#define VIRTCHNL2_EVENT_UNKNOWN 0
+#define VIRTCHNL2_EVENT_LINK_CHANGE 1
+/* These messages are only sent to PF from CP */
+#define VIRTCHNL2_EVENT_START_RESET_ADI 2
+#define VIRTCHNL2_EVENT_FINISH_RESET_ADI 3
+
+/* VIRTCHNL2_QUEUE_TYPE
+ * Transmit and Receive queue types are valid in legacy as well as split queue
+ * models. With Split Queue model, 2 additional types are introduced -
+ * TX_COMPLETION and RX_BUFFER. In split queue model, receive corresponds to
+ * the queue where hardware posts completions.
+ */
+#define VIRTCHNL2_QUEUE_TYPE_TX 0
+#define VIRTCHNL2_QUEUE_TYPE_RX 1
+#define VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION 2
+#define VIRTCHNL2_QUEUE_TYPE_RX_BUFFER 3
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_TX 4
+#define VIRTCHNL2_QUEUE_TYPE_CONFIG_RX 5
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX 6
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX 7
+#define VIRTCHNL2_QUEUE_TYPE_P2P_TX_COMPLETION 8
+#define VIRTCHNL2_QUEUE_TYPE_P2P_RX_BUFFER 9
+#define VIRTCHNL2_QUEUE_TYPE_MBX_TX 10
+#define VIRTCHNL2_QUEUE_TYPE_MBX_RX 11
+
+/* VIRTCHNL2_ITR_IDX
+ * Virtchannel interrupt throttling rate index
+ */
+#define VIRTCHNL2_ITR_IDX_0 0
+#define VIRTCHNL2_ITR_IDX_1 1
+#define VIRTCHNL2_ITR_IDX_2 2
+#define VIRTCHNL2_ITR_IDX_NO_ITR 3
+
+/* VIRTCHNL2_VECTOR_LIMITS
+ * Since PF/VF messages are limited by __le16 size, precalculate the maximum
+ * possible values of nested elements in virtchnl structures that virtual
+ * channel can possibly handle in a single message.
+ */
+
+#define VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_del_ena_dis_queues)) / \
+ sizeof(struct virtchnl2_queue_chunk))
+
+#define VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX (\
+ ((__le16)(~0) - sizeof(struct virtchnl2_queue_vector_maps)) / \
+ sizeof(struct virtchnl2_queue_vector))
+
+/* VIRTCHNL2_MAC_TYPE
+ * VIRTCHNL2_MAC_ADDR_PRIMARY
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_PRIMARY for the
+ * primary/device unicast MAC address filter for VIRTCHNL2_OP_ADD_MAC_ADDR and
+ * VIRTCHNL2_OP_DEL_MAC_ADDR. This allows for the underlying control plane
+ * function to accurately track the MAC address and for VM/function reset.
+ *
+ * VIRTCHNL2_MAC_ADDR_EXTRA
+ * PF/VF driver should set @type to VIRTCHNL2_MAC_ADDR_EXTRA for any extra
+ * unicast and/or multicast filters that are being added/deleted via
+ * VIRTCHNL2_OP_ADD_MAC_ADDR/VIRTCHNL2_OP_DEL_MAC_ADDR respectively.
+ */
+#define VIRTCHNL2_MAC_ADDR_PRIMARY 1
+#define VIRTCHNL2_MAC_ADDR_EXTRA 2
+
+/* VIRTCHNL2_PROMISC_FLAGS
+ * Flags used for promiscuous mode
+ */
+#define VIRTCHNL2_UNICAST_PROMISC BIT(0)
+#define VIRTCHNL2_MULTICAST_PROMISC BIT(1)
+
+/* VIRTCHNL2_PROTO_HDR_TYPE
+ * Protocol header type within a packet segment. A segment consists of one or
+ * more protocol headers that make up a logical group of protocol headers. Each
+ * logical group of protocol headers encapsulates or is encapsulated using/by
+ * tunneling or encapsulation protocols for network virtualization.
+ */
+/* VIRTCHNL2_PROTO_HDR_ANY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ANY 0
+#define VIRTCHNL2_PROTO_HDR_PRE_MAC 1
+/* VIRTCHNL2_PROTO_HDR_MAC is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_MAC 2
+#define VIRTCHNL2_PROTO_HDR_POST_MAC 3
+#define VIRTCHNL2_PROTO_HDR_ETHERTYPE 4
+#define VIRTCHNL2_PROTO_HDR_VLAN 5
+#define VIRTCHNL2_PROTO_HDR_SVLAN 6
+#define VIRTCHNL2_PROTO_HDR_CVLAN 7
+#define VIRTCHNL2_PROTO_HDR_MPLS 8
+#define VIRTCHNL2_PROTO_HDR_UMPLS 9
+#define VIRTCHNL2_PROTO_HDR_MMPLS 10
+#define VIRTCHNL2_PROTO_HDR_PTP 11
+#define VIRTCHNL2_PROTO_HDR_CTRL 12
+#define VIRTCHNL2_PROTO_HDR_LLDP 13
+#define VIRTCHNL2_PROTO_HDR_ARP 14
+#define VIRTCHNL2_PROTO_HDR_ECP 15
+#define VIRTCHNL2_PROTO_HDR_EAPOL 16
+#define VIRTCHNL2_PROTO_HDR_PPPOD 17
+#define VIRTCHNL2_PROTO_HDR_PPPOE 18
+/* VIRTCHNL2_PROTO_HDR_IPV4 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4 19
+/* IPv4 and IPv6 Fragment header types are only associated to
+ * VIRTCHNL2_PROTO_HDR_IPV4 and VIRTCHNL2_PROTO_HDR_IPV6 respectively,
+ * cannot be used independently.
+ */
+/* VIRTCHNL2_PROTO_HDR_IPV4_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV4_FRAG 20
+/* VIRTCHNL2_PROTO_HDR_IPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6 21
+/* VIRTCHNL2_PROTO_HDR_IPV6_FRAG is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_IPV6_FRAG 22
+#define VIRTCHNL2_PROTO_HDR_IPV6_EH 23
+/* VIRTCHNL2_PROTO_HDR_UDP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_UDP 24
+/* VIRTCHNL2_PROTO_HDR_TCP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TCP 25
+/* VIRTCHNL2_PROTO_HDR_SCTP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_SCTP 26
+/* VIRTCHNL2_PROTO_HDR_ICMP is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMP 27
+/* VIRTCHNL2_PROTO_HDR_ICMPV6 is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_ICMPV6 28
+#define VIRTCHNL2_PROTO_HDR_IGMP 29
+#define VIRTCHNL2_PROTO_HDR_AH 30
+#define VIRTCHNL2_PROTO_HDR_ESP 31
+#define VIRTCHNL2_PROTO_HDR_IKE 32
+#define VIRTCHNL2_PROTO_HDR_NATT_KEEP 33
+/* VIRTCHNL2_PROTO_HDR_PAY is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_PAY 34
+#define VIRTCHNL2_PROTO_HDR_L2TPV2 35
+#define VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL 36
+#define VIRTCHNL2_PROTO_HDR_L2TPV3 37
+#define VIRTCHNL2_PROTO_HDR_GTP 38
+#define VIRTCHNL2_PROTO_HDR_GTP_EH 39
+#define VIRTCHNL2_PROTO_HDR_GTPCV2 40
+#define VIRTCHNL2_PROTO_HDR_GTPC_TEID 41
+#define VIRTCHNL2_PROTO_HDR_GTPU 42
+#define VIRTCHNL2_PROTO_HDR_GTPU_UL 43
+#define VIRTCHNL2_PROTO_HDR_GTPU_DL 44
+#define VIRTCHNL2_PROTO_HDR_ECPRI 45
+#define VIRTCHNL2_PROTO_HDR_VRRP 46
+#define VIRTCHNL2_PROTO_HDR_OSPF 47
+/* VIRTCHNL2_PROTO_HDR_TUN is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_TUN 48
+#define VIRTCHNL2_PROTO_HDR_GRE 49
+#define VIRTCHNL2_PROTO_HDR_NVGRE 50
+#define VIRTCHNL2_PROTO_HDR_VXLAN 51
+#define VIRTCHNL2_PROTO_HDR_VXLAN_GPE 52
+#define VIRTCHNL2_PROTO_HDR_GENEVE 53
+#define VIRTCHNL2_PROTO_HDR_NSH 54
+#define VIRTCHNL2_PROTO_HDR_QUIC 55
+#define VIRTCHNL2_PROTO_HDR_PFCP 56
+#define VIRTCHNL2_PROTO_HDR_PFCP_NODE 57
+#define VIRTCHNL2_PROTO_HDR_PFCP_SESSION 58
+#define VIRTCHNL2_PROTO_HDR_RTP 59
+#define VIRTCHNL2_PROTO_HDR_ROCE 60
+#define VIRTCHNL2_PROTO_HDR_ROCEV1 61
+#define VIRTCHNL2_PROTO_HDR_ROCEV2 62
+/* protocol ids up to 32767 are reserved for AVF use */
+/* 32768 - 65534 are used for user defined protocol ids */
+/* VIRTCHNL2_PROTO_HDR_NO_PROTO is a mandatory protocol id */
+#define VIRTCHNL2_PROTO_HDR_NO_PROTO 65535
+
+#define VIRTCHNL2_VERSION_MAJOR_2 2
+#define VIRTCHNL2_VERSION_MINOR_0 0
+
+
+/* VIRTCHNL2_OP_VERSION
+ * VF posts its version number to the CP. CP responds with its version number
+ * in the same format, along with a return code.
+ * If there is a major version mismatch, then the VF cannot operate.
+ * If there is a minor version mismatch, then the VF can operate but should
+ * add a warning to the system log.
+ *
+ * This version opcode MUST always be specified as == 1, regardless of other
+ * changes in the API. The CP must always respond to this message without
+ * error regardless of version mismatch.
+ */
+struct virtchnl2_version_info {
+ u32 major;
+ u32 minor;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_version_info);
+
+/* VIRTCHNL2_OP_GET_CAPS
+ * Dataplane driver sends this message to CP to negotiate capabilities and
+ * provides a virtchnl2_get_capabilities structure with its desired
+ * capabilities, max_sriov_vfs and num_allocated_vectors.
+ * CP responds with a virtchnl2_get_capabilities structure updated
+ * with allowed capabilities and the other fields as below.
+ * If PF sets max_sriov_vfs as 0, CP will respond with max number of VFs
+ * that can be created by this PF. For any other value 'n', CP responds
+ * with max_sriov_vfs set to min(n, x) where x is the max number of VFs
+ * allowed by CP's policy. max_sriov_vfs is not applicable for VFs.
+ * If dataplane driver sets num_allocated_vectors as 0, CP will respond with 1
+ * which is default vector associated with the default mailbox. For any other
+ * value 'n', CP responds with a value <= n based on the CP's policy of
+ * max number of vectors for a PF.
+ * CP will respond with the vector ID of mailbox allocated to the PF in
+ * mailbox_vector_id and the number of itr index registers in itr_idx_map.
+ * It also responds with default number of vports that the dataplane driver
+ * should comeup with in default_num_vports and maximum number of vports that
+ * can be supported in max_vports
+ */
+struct virtchnl2_get_capabilities {
+ /* see VIRTCHNL2_CHECKSUM_OFFLOAD_CAPS definitions */
+ __le32 csum_caps;
+
+ /* see VIRTCHNL2_SEGMENTATION_OFFLOAD_CAPS definitions */
+ __le32 seg_caps;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 hsplit_caps;
+
+ /* see VIRTCHNL2_RSC_OFFLOAD_CAPS definitions */
+ __le32 rsc_caps;
+
+ /* see VIRTCHNL2_RSS_FLOW_TYPE_CAPS definitions */
+ __le64 rss_caps;
+
+
+ /* see VIRTCHNL2_OTHER_CAPS definitions */
+ __le64 other_caps;
+
+ /* DYN_CTL register offset and vector id for mailbox provided by CP */
+ __le32 mailbox_dyn_ctl;
+ __le16 mailbox_vector_id;
+ /* Maximum number of allocated vectors for the device */
+ __le16 num_allocated_vectors;
+
+ /* Maximum number of queues that can be supported */
+ __le16 max_rx_q;
+ __le16 max_tx_q;
+ __le16 max_rx_bufq;
+ __le16 max_tx_complq;
+
+ /* The PF sends the maximum VFs it is requesting. The CP responds with
+ * the maximum VFs granted.
+ */
+ __le16 max_sriov_vfs;
+
+ /* maximum number of vports that can be supported */
+ __le16 max_vports;
+ /* default number of vports driver should allocate on load */
+ __le16 default_num_vports;
+
+ /* Max header length hardware can parse/checksum, in bytes */
+ __le16 max_tx_hdr_size;
+
+ /* Max number of scatter gather buffers that can be sent per transmit
+ * packet without needing to be linearized
+ */
+ u8 max_sg_bufs_per_tx_pkt;
+
+ /* see VIRTCHNL2_ITR_IDX definition */
+ u8 itr_idx_map;
+
+ __le16 pad1;
+
+ /* version of Control Plane that is running */
+ __le16 oem_cp_ver_major;
+ __le16 oem_cp_ver_minor;
+ /* see VIRTCHNL2_DEVICE_TYPE definitions */
+ __le32 device_type;
+
+ u8 reserved[12];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(80, virtchnl2_get_capabilities);
+
+struct virtchnl2_queue_reg_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ __le32 pad;
+
+ /* Queue tail register offset and spacing provided by CP */
+ __le64 qtail_reg_start;
+ __le32 qtail_reg_spacing;
+
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_queue_reg_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_reg_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_reg_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_reg_chunks);
+
+#define VIRTCHNL2_ETH_LENGTH_OF_ADDRESS 6
+
+/* VIRTCHNL2_OP_CREATE_VPORT
+ * PF sends this message to CP to create a vport by filling in required
+ * fields of virtchnl2_create_vport structure.
+ * CP responds with the updated virtchnl2_create_vport structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_vport {
+ /* PF/VF populates the following fields on request */
+ /* see VIRTCHNL2_VPORT_TYPE definitions */
+ __le16 vport_type;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 txq_model;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 rxq_model;
+ __le16 num_tx_q;
+ /* valid only if txq_model is split queue */
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ /* valid only if rxq_model is split queue */
+ __le16 num_rx_bufq;
+ /* relative receive queue index to be used as default */
+ __le16 default_rx_q;
+ /* used to align PF and CP in case of default multiple vports, it is
+ * filled by the PF and CP returns the same value, to enable the driver
+ * to support multiple asynchronous parallel CREATE_VPORT requests and
+ * associate a response to a specific request
+ */
+ __le16 vport_index;
+
+ /* CP populates the following fields on response */
+ __le16 max_mtu;
+ __le32 vport_id;
+ u8 default_mac_addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ __le16 pad;
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 rx_desc_ids;
+ /* see VIRTCHNL2_TX_DESC_IDS definitions */
+ __le64 tx_desc_ids;
+
+#define MAX_Q_REGIONS 16
+ __le32 max_qs_per_qregion[MAX_Q_REGIONS];
+ __le32 qregion_total_qs;
+ __le16 qregion_type;
+ __le16 pad2;
+
+ /* see VIRTCHNL2_RSS_ALGORITHM definitions */
+ __le32 rss_algorithm;
+ __le16 rss_key_size;
+ __le16 rss_lut_size;
+
+ /* see VIRTCHNL2_HEADER_SPLIT_CAPS definitions */
+ __le32 rx_split_pos;
+
+ u8 reserved[20];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(192, virtchnl2_create_vport);
+
+/* VIRTCHNL2_OP_DESTROY_VPORT
+ * VIRTCHNL2_OP_ENABLE_VPORT
+ * VIRTCHNL2_OP_DISABLE_VPORT
+ * PF sends this message to CP to destroy, enable or disable a vport by filling
+ * in the vport_id in virtchnl2_vport structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_vport {
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_vport);
+
+/* Transmit queue config info */
+struct virtchnl2_txq_info {
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+
+ __le32 queue_id;
+ /* valid only if queue model is split and type is transmit queue. Used
+ * in many to one mapping of transmit queues to completion queue
+ */
+ __le16 relative_queue_id;
+
+ /* see VIRTCHNL2_QUEUE_MODEL definitions */
+ __le16 model;
+
+ /* see VIRTCHNL2_TXQ_SCHED_MODE definitions */
+ __le16 sched_mode;
+
+ /* see VIRTCHNL2_TXQ_FLAGS definitions */
+ __le16 qflags;
+ __le16 ring_len;
+
+ /* valid only if queue model is split and type is transmit queue */
+ __le16 tx_compl_queue_id;
+ /* valid only if queue type is VIRTCHNL2_QUEUE_TYPE_MAILBOX_TX */
+ /* see VIRTCHNL2_PEER_TYPE definitions */
+ __le16 peer_type;
+ /* valid only if queue type is CONFIG_TX and used to deliver messages
+ * for the respective CONFIG_TX queue
+ */
+ __le16 peer_rx_queue_id;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+ u8 pad[2];
+
+ /* Egress pasid is used for SIOV use case */
+ __le32 egress_pasid;
+ __le32 egress_hdr_pasid;
+ __le32 egress_buf_pasid;
+
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_txq_info);
+
+/* VIRTCHNL2_OP_CONFIG_TX_QUEUES
+ * PF sends this message to set up parameters for one or more transmit queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_txq_info
+ * structures. CP configures requested queues and returns a status code. If
+ * num_qinfo specified is greater than the number of queues associated with the
+ * vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_tx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[10];
+ struct virtchnl2_txq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(72, virtchnl2_config_tx_queues);
+
+/* Receive queue config info */
+struct virtchnl2_rxq_info {
+ /* see VIRTCHNL2_RX_DESC_IDS definitions */
+ __le64 desc_ids;
+ __le64 dma_ring_addr;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 queue_id;
+
+ /* see QUEUE_MODEL definitions */
+ __le16 model;
+
+ __le16 hdr_buffer_size;
+ __le32 data_buffer_size;
+ __le32 max_pkt_size;
+
+ __le16 ring_len;
+ u8 buffer_notif_stride;
+ u8 pad[1];
+
+ /* Applicable only for receive buffer queues */
+ __le64 dma_head_wb_addr;
+
+ /* Applicable only for receive completion queues */
+ /* see VIRTCHNL2_RXQ_FLAGS definitions */
+ __le16 qflags;
+
+ __le16 rx_buffer_low_watermark;
+
+ /* valid only in split queue model */
+ __le16 rx_bufq1_id;
+ /* valid only in split queue model */
+ __le16 rx_bufq2_id;
+ /* it indicates if there is a second buffer, rx_bufq2_id is valid only
+ * if this field is set
+ */
+ u8 bufq2_ena;
+ u8 pad2;
+
+ /* value ranges from 0 to 15 */
+ __le16 qregion_id;
+
+ /* Ingress pasid is used for SIOV use case */
+ __le32 ingress_pasid;
+ __le32 ingress_hdr_pasid;
+ __le32 ingress_buf_pasid;
+
+ u8 reserved[16];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(88, virtchnl2_rxq_info);
+
+/* VIRTCHNL2_OP_CONFIG_RX_QUEUES
+ * PF sends this message to set up parameters for one or more receive queues.
+ * This message contains an array of num_qinfo instances of virtchnl2_rxq_info
+ * structures. CP configures requested queues and returns a status code.
+ * If the number of queues specified is greater than the number of queues
+ * associated with the vport, an error is returned and no queues are configured.
+ */
+struct virtchnl2_config_rx_queues {
+ __le32 vport_id;
+ __le16 num_qinfo;
+
+ u8 reserved[18];
+ struct virtchnl2_rxq_info qinfo[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(112, virtchnl2_config_rx_queues);
+
+/* VIRTCHNL2_OP_ADD_QUEUES
+ * PF sends this message to request additional transmit/receive queues beyond
+ * the ones that were assigned via CREATE_VPORT request. virtchnl2_add_queues
+ * structure is used to specify the number of each type of queues.
+ * CP responds with the same structure with the actual number of queues assigned
+ * followed by num_chunks of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_add_queues {
+ __le32 vport_id;
+ __le16 num_tx_q;
+ __le16 num_tx_complq;
+ __le16 num_rx_q;
+ __le16 num_rx_bufq;
+ u8 reserved[4];
+ struct virtchnl2_queue_reg_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(56, virtchnl2_add_queues);
+
+/* Structure to specify a chunk of contiguous interrupt vectors */
+struct virtchnl2_vector_chunk {
+ __le16 start_vector_id;
+ __le16 start_evv_id;
+ __le16 num_vectors;
+ __le16 pad1;
+
+ /* Register offsets and spacing provided by CP.
+ * dynamic control registers are used for enabling/disabling/re-enabling
+ * interrupts and updating interrupt rates in the hotpath. Any changes
+ * to interrupt rates in the dynamic control registers will be reflected
+ * in the interrupt throttling rate registers.
+ * itrn registers are used to update interrupt rates for specific
+ * interrupt indices without modifying the state of the interrupt.
+ */
+ __le32 dynctl_reg_start;
+ __le32 dynctl_reg_spacing;
+
+ __le32 itrn_reg_start;
+ __le32 itrn_reg_spacing;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_vector_chunk);
+
+/* Structure to specify several chunks of contiguous interrupt vectors */
+struct virtchnl2_vector_chunks {
+ __le16 num_vchunks;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunk vchunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(48, virtchnl2_vector_chunks);
+
+/* VIRTCHNL2_OP_ALLOC_VECTORS
+ * PF sends this message to request additional interrupt vectors beyond the
+ * ones that were assigned via GET_CAPS request. virtchnl2_alloc_vectors
+ * structure is used to specify the number of vectors requested. CP responds
+ * with the same structure with the actual number of vectors assigned followed
+ * by virtchnl2_vector_chunks structure identifying the vector ids.
+ */
+struct virtchnl2_alloc_vectors {
+ __le16 num_vectors;
+ u8 reserved[14];
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(64, virtchnl2_alloc_vectors);
+
+/* VIRTCHNL2_OP_DEALLOC_VECTORS
+ * PF sends this message to release the vectors.
+ * PF sends virtchnl2_vector_chunks struct to specify the vectors it is giving
+ * away. CP performs requested action and returns status.
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_LUT
+ * VIRTCHNL2_OP_SET_RSS_LUT
+ * PF sends this message to get or set RSS lookup table. Only supported if
+ * both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_lut structure
+ */
+struct virtchnl2_rss_lut {
+ __le32 vport_id;
+ __le16 lut_entries_start;
+ __le16 lut_entries;
+ u8 reserved[4];
+ __le32 lut[1]; /* RSS lookup table */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_lut);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * PF sends this message to get RSS key. Only supported if both PF and CP
+ * drivers set the VIRTCHNL2_CAP_RSS bit during configuration negotiation. Uses
+ * the virtchnl2_rss_key structure
+ */
+
+/* VIRTCHNL2_OP_GET_RSS_HASH
+ * VIRTCHNL2_OP_SET_RSS_HASH
+ * PF sends these messages to get and set the hash filter enable bits for RSS.
+ * By default, the CP sets these to all possible traffic types that the
+ * hardware supports. The PF can query this value if it wants to change the
+ * traffic types that are hashed by the hardware.
+ * Only supported if both PF and CP drivers set the VIRTCHNL2_CAP_RSS bit
+ * during configuration negotiation.
+ */
+struct virtchnl2_rss_hash {
+ /* Packet Type Groups bitmap */
+ __le64 ptype_groups;
+ __le32 vport_id;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_rss_hash);
+
+/* VIRTCHNL2_OP_SET_SRIOV_VFS
+ * This message is used to set number of SRIOV VFs to be created. The actual
+ * allocation of resources for the VFs in terms of vport, queues and interrupts
+ * is done by CP. When this call completes, the APF driver calls
+ * pci_enable_sriov to let the OS instantiate the SRIOV PCIE devices.
+ * The number of VFs set to 0 will destroy all the VFs of this function.
+ */
+
+struct virtchnl2_sriov_vfs_info {
+ __le16 num_vfs;
+ __le16 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_sriov_vfs_info);
+
+/* VIRTCHNL2_OP_CREATE_ADI
+ * PF sends this message to CP to create ADI by filling in required
+ * fields of virtchnl2_create_adi structure.
+ * CP responds with the updated virtchnl2_create_adi structure containing the
+ * necessary fields followed by chunks which in turn will have an array of
+ * num_chunks entries of virtchnl2_queue_chunk structures.
+ */
+struct virtchnl2_create_adi {
+ /* PF sends PASID to CP */
+ __le32 pasid;
+ /*
+ * mbx_id is set to 1 by PF when requesting CP to provide HW mailbox
+ * id else it is set to 0 by PF
+ */
+ __le16 mbx_id;
+ /* PF sends mailbox vector id to CP */
+ __le16 mbx_vec_id;
+ /* CP populates ADI id */
+ __le16 adi_id;
+ u8 reserved[64];
+ u8 pad[6];
+ /* CP populates queue chunks */
+ struct virtchnl2_queue_reg_chunks chunks;
+ /* PF sends vector chunks to CP */
+ struct virtchnl2_vector_chunks vchunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(168, virtchnl2_create_adi);
+
+/* VIRTCHNL2_OP_DESTROY_ADI
+ * PF sends this message to CP to destroy ADI by filling
+ * in the adi_id in virtchnl2_destropy_adi structure.
+ * CP responds with the status of the requested operation.
+ */
+struct virtchnl2_destroy_adi {
+ __le16 adi_id;
+ u8 reserved[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(4, virtchnl2_destroy_adi);
+
+/* Based on the descriptor type the PF supports, CP fills ptype_id_10 or
+ * ptype_id_8 for flex and base descriptor respectively. If ptype_id_10 value
+ * is set to 0xFFFF, PF should consider this ptype as dummy one and it is the
+ * last ptype.
+ */
+struct virtchnl2_ptype {
+ __le16 ptype_id_10;
+ u8 ptype_id_8;
+ /* number of protocol ids the packet supports, maximum of 32
+ * protocol ids are supported
+ */
+ u8 proto_id_count;
+ __le16 pad;
+ /* proto_id_count decides the allocation of protocol id array */
+ /* see VIRTCHNL2_PROTO_HDR_TYPE */
+ __le16 proto_id[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_ptype);
+
+/* VIRTCHNL2_OP_GET_PTYPE_INFO
+ * PF sends this message to CP to get all supported packet types. It does by
+ * filling in start_ptype_id and num_ptypes. Depending on descriptor type the
+ * PF supports, it sets num_ptypes to 1024 (10-bit ptype) for flex descriptor
+ * and 256 (8-bit ptype) for base descriptor support. CP responds back to PF by
+ * populating start_ptype_id, num_ptypes and array of ptypes. If all ptypes
+ * doesn't fit into one mailbox buffer, CP splits ptype info into multiple
+ * messages, where each message will have the start ptype id, number of ptypes
+ * sent in that message and the ptype array itself. When CP is done updating
+ * all ptype information it extracted from the package (number of ptypes
+ * extracted might be less than what PF expects), it will append a dummy ptype
+ * (which has 'ptype_id_10' of 'struct virtchnl2_ptype' as 0xFFFF) to the ptype
+ * array. PF is expected to receive multiple VIRTCHNL2_OP_GET_PTYPE_INFO
+ * messages.
+ */
+struct virtchnl2_get_ptype_info {
+ __le16 start_ptype_id;
+ __le16 num_ptypes;
+ __le32 pad;
+ struct virtchnl2_ptype ptype[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_get_ptype_info);
+
+/* VIRTCHNL2_OP_GET_STATS
+ * PF/VF sends this message to CP to get the update stats by specifying the
+ * vport_id. CP responds with stats in struct virtchnl2_vport_stats.
+ */
+struct virtchnl2_vport_stats {
+ __le32 vport_id;
+ u8 pad[4];
+
+ __le64 rx_bytes; /* received bytes */
+ __le64 rx_unicast; /* received unicast pkts */
+ __le64 rx_multicast; /* received multicast pkts */
+ __le64 rx_broadcast; /* received broadcast pkts */
+ __le64 rx_discards;
+ __le64 rx_errors;
+ __le64 rx_unknown_protocol;
+ __le64 tx_bytes; /* transmitted bytes */
+ __le64 tx_unicast; /* transmitted unicast pkts */
+ __le64 tx_multicast; /* transmitted multicast pkts */
+ __le64 tx_broadcast; /* transmitted broadcast pkts */
+ __le64 tx_discards;
+ __le64 tx_errors;
+ __le64 rx_invalid_frame_length;
+ __le64 rx_overflow_drop;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(128, virtchnl2_vport_stats);
+
+/* VIRTCHNL2_OP_EVENT
+ * CP sends this message to inform the PF/VF driver of events that may affect
+ * it. No direct response is expected from the driver, though it may generate
+ * other messages in response to this one.
+ */
+struct virtchnl2_event {
+ /* see VIRTCHNL2_EVENT_CODES definitions */
+ __le32 event;
+ /* link_speed provided in Mbps */
+ __le32 link_speed;
+ __le32 vport_id;
+ u8 link_status;
+ u8 pad[1];
+ /* CP sends reset notification to PF with corresponding ADI ID */
+ __le16 adi_id;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_event);
+
+/* VIRTCHNL2_OP_GET_RSS_KEY
+ * VIRTCHNL2_OP_SET_RSS_KEY
+ * PF/VF sends this message to get or set RSS key. Only supported if both
+ * PF/VF and CP drivers set the VIRTCHNL2_CAP_RSS bit during configuration
+ * negotiation. Uses the virtchnl2_rss_key structure
+ */
+struct virtchnl2_rss_key {
+ __le32 vport_id;
+ __le16 key_len;
+ u8 pad;
+ u8 key[1]; /* RSS hash key, packed bytes */
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_rss_key);
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl2_queue_chunk {
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 type;
+ __le32 start_queue_id;
+ __le32 num_queues;
+ u8 reserved[4];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_queue_chunk);
+
+/* structure to specify several chunks of contiguous queues */
+struct virtchnl2_queue_chunks {
+ __le16 num_chunks;
+ u8 reserved[6];
+ struct virtchnl2_queue_chunk chunks[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_chunks);
+
+/* VIRTCHNL2_OP_ENABLE_QUEUES
+ * VIRTCHNL2_OP_DISABLE_QUEUES
+ * VIRTCHNL2_OP_DEL_QUEUES
+ *
+ * PF sends these messages to enable, disable or delete queues specified in
+ * chunks. PF sends virtchnl2_del_ena_dis_queues struct to specify the queues
+ * to be enabled/disabled/deleted. Also applicable to single queue receive or
+ * transmit. CP performs requested action and returns status.
+ */
+struct virtchnl2_del_ena_dis_queues {
+ __le32 vport_id;
+ u8 reserved[4];
+ struct virtchnl2_queue_chunks chunks;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(32, virtchnl2_del_ena_dis_queues);
+
+/* Queue to vector mapping */
+struct virtchnl2_queue_vector {
+ __le32 queue_id;
+ __le16 vector_id;
+ u8 pad[2];
+
+ /* see VIRTCHNL2_ITR_IDX definitions */
+ __le32 itr_idx;
+
+ /* see VIRTCHNL2_QUEUE_TYPE definitions */
+ __le32 queue_type;
+ u8 reserved[8];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(24, virtchnl2_queue_vector);
+
+/* VIRTCHNL2_OP_MAP_QUEUE_VECTOR
+ * VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR
+ *
+ * PF sends this message to map or unmap queues to vectors and interrupt
+ * throttling rate index registers. External data buffer contains
+ * virtchnl2_queue_vector_maps structure that contains num_qv_maps of
+ * virtchnl2_queue_vector structures. CP maps the requested queue vector maps
+ * after validating the queue and vector ids and returns a status code.
+ */
+struct virtchnl2_queue_vector_maps {
+ __le32 vport_id;
+ __le16 num_qv_maps;
+ u8 pad[10];
+ struct virtchnl2_queue_vector qv_maps[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(40, virtchnl2_queue_vector_maps);
+
+/* VIRTCHNL2_OP_LOOPBACK
+ *
+ * PF/VF sends this message to transition to/from the loopback state. Setting
+ * the 'enable' to 1 enables the loopback state and setting 'enable' to 0
+ * disables it. CP configures the state to loopback and returns status.
+ */
+struct virtchnl2_loopback {
+ __le32 vport_id;
+ u8 enable;
+ u8 pad[3];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_loopback);
+
+/* structure to specify each MAC address */
+struct virtchnl2_mac_addr {
+ u8 addr[VIRTCHNL2_ETH_LENGTH_OF_ADDRESS];
+ /* see VIRTCHNL2_MAC_TYPE definitions */
+ u8 type;
+ u8 pad;
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_mac_addr);
+
+/* VIRTCHNL2_OP_ADD_MAC_ADDR
+ * VIRTCHNL2_OP_DEL_MAC_ADDR
+ *
+ * PF/VF driver uses this structure to send list of MAC addresses to be
+ * added/deleted to the CP where as CP performs the action and returns the
+ * status.
+ */
+struct virtchnl2_mac_addr_list {
+ __le32 vport_id;
+ __le16 num_mac_addr;
+ u8 pad[2];
+ struct virtchnl2_mac_addr mac_addr_list[1];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(16, virtchnl2_mac_addr_list);
+
+/* VIRTCHNL2_OP_CONFIG_PROMISCUOUS_MODE
+ *
+ * PF/VF sends vport id and flags to the CP where as CP performs the action
+ * and returns the status.
+ */
+struct virtchnl2_promisc_info {
+ __le32 vport_id;
+ /* see VIRTCHNL2_PROMISC_FLAGS definitions */
+ __le16 flags;
+ u8 pad[2];
+};
+
+VIRTCHNL2_CHECK_STRUCT_LEN(8, virtchnl2_promisc_info);
+
+
+static inline const char *virtchnl2_op_str(__le32 v_opcode)
+{
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ return "VIRTCHNL2_OP_VERSION";
+ case VIRTCHNL2_OP_GET_CAPS:
+ return "VIRTCHNL2_OP_GET_CAPS";
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ return "VIRTCHNL2_OP_CREATE_VPORT";
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ return "VIRTCHNL2_OP_DESTROY_VPORT";
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ return "VIRTCHNL2_OP_ENABLE_VPORT";
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ return "VIRTCHNL2_OP_DISABLE_VPORT";
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_TX_QUEUES";
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ return "VIRTCHNL2_OP_CONFIG_RX_QUEUES";
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ return "VIRTCHNL2_OP_ENABLE_QUEUES";
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ return "VIRTCHNL2_OP_DISABLE_QUEUES";
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ return "VIRTCHNL2_OP_ADD_QUEUES";
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ return "VIRTCHNL2_OP_DEL_QUEUES";
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_MAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ return "VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR";
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ return "VIRTCHNL2_OP_GET_RSS_KEY";
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ return "VIRTCHNL2_OP_SET_RSS_KEY";
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ return "VIRTCHNL2_OP_GET_RSS_LUT";
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ return "VIRTCHNL2_OP_SET_RSS_LUT";
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ return "VIRTCHNL2_OP_GET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ return "VIRTCHNL2_OP_SET_RSS_HASH";
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ return "VIRTCHNL2_OP_SET_SRIOV_VFS";
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ return "VIRTCHNL2_OP_ALLOC_VECTORS";
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ return "VIRTCHNL2_OP_DEALLOC_VECTORS";
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ return "VIRTCHNL2_OP_GET_PTYPE_INFO";
+ case VIRTCHNL2_OP_GET_STATS:
+ return "VIRTCHNL2_OP_GET_STATS";
+ case VIRTCHNL2_OP_EVENT:
+ return "VIRTCHNL2_OP_EVENT";
+ case VIRTCHNL2_OP_RESET_VF:
+ return "VIRTCHNL2_OP_RESET_VF";
+ case VIRTCHNL2_OP_CREATE_ADI:
+ return "VIRTCHNL2_OP_CREATE_ADI";
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ return "VIRTCHNL2_OP_DESTROY_ADI";
+ default:
+ return "Unsupported (update virtchnl2.h)";
+ }
+}
+
+/**
+ * virtchnl2_vc_validate_vf_msg
+ * @ver: Virtchnl2 version info
+ * @v_opcode: Opcode for the message
+ * @msg: pointer to the msg buffer
+ * @msglen: msg length
+ *
+ * validate msg format against struct for each opcode
+ */
+static inline int
+virtchnl2_vc_validate_vf_msg(__rte_unused struct virtchnl2_version_info *ver, u32 v_opcode,
+ u8 *msg, __le16 msglen)
+{
+ bool err_msg_format = false;
+ __le32 valid_len = 0;
+
+ /* Validate message length. */
+ switch (v_opcode) {
+ case VIRTCHNL2_OP_VERSION:
+ valid_len = sizeof(struct virtchnl2_version_info);
+ break;
+ case VIRTCHNL2_OP_GET_CAPS:
+ valid_len = sizeof(struct virtchnl2_get_capabilities);
+ break;
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ valid_len = sizeof(struct virtchnl2_create_vport);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_vport *cvport =
+ (struct virtchnl2_create_vport *)msg;
+
+ if (cvport->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (cvport->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_CREATE_ADI:
+ valid_len = sizeof(struct virtchnl2_create_adi);
+ if (msglen >= valid_len) {
+ struct virtchnl2_create_adi *cadi =
+ (struct virtchnl2_create_adi *)msg;
+
+ if (cadi->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ if (cadi->vchunks.num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (cadi->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ valid_len += (cadi->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DESTROY_ADI:
+ valid_len = sizeof(struct virtchnl2_destroy_adi);
+ break;
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ valid_len = sizeof(struct virtchnl2_vport);
+ break;
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_tx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_tx_queues *ctq =
+ (struct virtchnl2_config_tx_queues *)msg;
+ if (ctq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (ctq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ valid_len = sizeof(struct virtchnl2_config_rx_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_config_rx_queues *crq =
+ (struct virtchnl2_config_rx_queues *)msg;
+ if (crq->num_qinfo == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (crq->num_qinfo - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ }
+ break;
+ case VIRTCHNL2_OP_ADD_QUEUES:
+ valid_len = sizeof(struct virtchnl2_add_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_add_queues *add_q =
+ (struct virtchnl2_add_queues *)msg;
+
+ if (add_q->chunks.num_chunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (add_q->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_reg_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
+ case VIRTCHNL2_OP_DEL_QUEUES:
+ valid_len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl2_del_ena_dis_queues *qs =
+ (struct virtchnl2_del_ena_dis_queues *)msg;
+ if (qs->chunks.num_chunks == 0 ||
+ qs->chunks.num_chunks > VIRTCHNL2_OP_DEL_ENABLE_DISABLE_QUEUES_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (qs->chunks.num_chunks - 1) *
+ sizeof(struct virtchnl2_queue_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ valid_len = sizeof(struct virtchnl2_queue_vector_maps);
+ if (msglen >= valid_len) {
+ struct virtchnl2_queue_vector_maps *v_qp =
+ (struct virtchnl2_queue_vector_maps *)msg;
+ if (v_qp->num_qv_maps == 0 ||
+ v_qp->num_qv_maps > VIRTCHNL2_OP_MAP_UNMAP_QUEUE_VECTOR_MAX) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_qp->num_qv_maps - 1) *
+ sizeof(struct virtchnl2_queue_vector);
+ }
+ break;
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_alloc_vectors);
+ if (msglen >= valid_len) {
+ struct virtchnl2_alloc_vectors *v_av =
+ (struct virtchnl2_alloc_vectors *)msg;
+
+ if (v_av->vchunks.num_vchunks == 0) {
+ /* zero chunks is allowed as input */
+ break;
+ }
+
+ valid_len += (v_av->vchunks.num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
+ valid_len = sizeof(struct virtchnl2_vector_chunks);
+ if (msglen >= valid_len) {
+ struct virtchnl2_vector_chunks *v_chunks =
+ (struct virtchnl2_vector_chunks *)msg;
+ if (v_chunks->num_vchunks == 0) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len += (v_chunks->num_vchunks - 1) *
+ sizeof(struct virtchnl2_vector_chunk);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ valid_len = sizeof(struct virtchnl2_rss_key);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_key *vrk =
+ (struct virtchnl2_rss_key *)msg;
+
+ if (vrk->key_len == 0) {
+ /* zero length is allowed as input */
+ break;
+ }
+
+ valid_len += vrk->key_len - 1;
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ valid_len = sizeof(struct virtchnl2_rss_lut);
+ if (msglen >= valid_len) {
+ struct virtchnl2_rss_lut *vrl =
+ (struct virtchnl2_rss_lut *)msg;
+
+ if (vrl->lut_entries == 0) {
+ /* zero entries is allowed as input */
+ break;
+ }
+
+ valid_len += (vrl->lut_entries - 1) * sizeof(vrl->lut);
+ }
+ break;
+ case VIRTCHNL2_OP_GET_RSS_HASH:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
+ valid_len = sizeof(struct virtchnl2_rss_hash);
+ break;
+ case VIRTCHNL2_OP_SET_SRIOV_VFS:
+ valid_len = sizeof(struct virtchnl2_sriov_vfs_info);
+ break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ valid_len = sizeof(struct virtchnl2_get_ptype_info);
+ break;
+ case VIRTCHNL2_OP_GET_STATS:
+ valid_len = sizeof(struct virtchnl2_vport_stats);
+ break;
+ case VIRTCHNL2_OP_RESET_VF:
+ break;
+ /* These are always errors coming from the VF. */
+ case VIRTCHNL2_OP_EVENT:
+ case VIRTCHNL2_OP_UNKNOWN:
+ default:
+ return VIRTCHNL2_STATUS_ERR_PARAM;
+ }
+ /* few more checks */
+ if (err_msg_format || valid_len != msglen)
+ return VIRTCHNL2_STATUS_ERR_OPCODE_MISMATCH;
+
+ return 0;
+}
+
+#endif /* _VIRTCHNL_2_H_ */
diff --git a/drivers/common/idpf/base/virtchnl2_lan_desc.h b/drivers/common/idpf/base/virtchnl2_lan_desc.h
new file mode 100644
index 0000000000..b8cb22e474
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl2_lan_desc.h
@@ -0,0 +1,606 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+/*
+ * Copyright (C) 2019 Intel Corporation
+ *
+ * For licensing information, see the file 'LICENSE' in the root folder
+ */
+#ifndef _VIRTCHNL2_LAN_DESC_H_
+#define _VIRTCHNL2_LAN_DESC_H_
+
+/* VIRTCHNL2_TX_DESC_IDS
+ * Transmit descriptor ID flags
+ */
+#define VIRTCHNL2_TXDID_DATA BIT(0)
+#define VIRTCHNL2_TXDID_CTX BIT(1)
+#define VIRTCHNL2_TXDID_REINJECT_CTX BIT(2)
+#define VIRTCHNL2_TXDID_FLEX_DATA BIT(3)
+#define VIRTCHNL2_TXDID_FLEX_CTX BIT(4)
+#define VIRTCHNL2_TXDID_FLEX_TSO_CTX BIT(5)
+#define VIRTCHNL2_TXDID_FLEX_TSYN_L2TAG1 BIT(6)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG1_L2TAG2 BIT(7)
+#define VIRTCHNL2_TXDID_FLEX_TSO_L2TAG2_PARSTAG_CTX BIT(8)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_TSO_CTX BIT(9)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_SA_CTX BIT(10)
+#define VIRTCHNL2_TXDID_FLEX_L2TAG2_CTX BIT(11)
+#define VIRTCHNL2_TXDID_FLEX_FLOW_SCHED BIT(12)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_TSO_CTX BIT(13)
+#define VIRTCHNL2_TXDID_FLEX_HOSTSPLIT_CTX BIT(14)
+#define VIRTCHNL2_TXDID_DESC_DONE BIT(15)
+
+/* VIRTCHNL2_RX_DESC_IDS
+ * Receive descriptor IDs (range from 0 to 63)
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE 0
+#define VIRTCHNL2_RXDID_1_32B_BASE 1
+/* FLEX_SQ_NIC and FLEX_SPLITQ share desc ids because they can be
+ * differentiated based on queue model; e.g. single queue model can
+ * only use FLEX_SQ_NIC and split queue model can only use FLEX_SPLITQ
+ * for DID 2.
+ */
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ 2
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC 2
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW 3
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB 4
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL 5
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2 6
+#define VIRTCHNL2_RXDID_7_HW_RSVD 7
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC 16
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN 17
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4 18
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6 19
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW 20
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP 21
+/* 22 through 63 are reserved */
+
+/* VIRTCHNL2_RX_DESC_ID_BITMASKS
+ * Receive descriptor ID bitmasks
+ */
+#define VIRTCHNL2_RXDID_0_16B_BASE_M BIT(VIRTCHNL2_RXDID_0_16B_BASE)
+#define VIRTCHNL2_RXDID_1_32B_BASE_M BIT(VIRTCHNL2_RXDID_1_32B_BASE)
+#define VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M BIT(VIRTCHNL2_RXDID_2_FLEX_SPLITQ)
+#define VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M BIT(VIRTCHNL2_RXDID_2_FLEX_SQ_NIC)
+#define VIRTCHNL2_RXDID_3_FLEX_SQ_SW_M BIT(VIRTCHNL2_RXDID_3_FLEX_SQ_SW)
+#define VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB_M BIT(VIRTCHNL2_RXDID_4_FLEX_SQ_NIC_VEB)
+#define VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL_M BIT(VIRTCHNL2_RXDID_5_FLEX_SQ_NIC_ACL)
+#define VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2_M BIT(VIRTCHNL2_RXDID_6_FLEX_SQ_NIC_2)
+#define VIRTCHNL2_RXDID_7_HW_RSVD_M BIT(VIRTCHNL2_RXDID_7_HW_RSVD)
+/* 9 through 15 are reserved */
+#define VIRTCHNL2_RXDID_16_COMMS_GENERIC_M BIT(VIRTCHNL2_RXDID_16_COMMS_GENERIC)
+#define VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN_M BIT(VIRTCHNL2_RXDID_17_COMMS_AUX_VLAN)
+#define VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4_M BIT(VIRTCHNL2_RXDID_18_COMMS_AUX_IPV4)
+#define VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6_M BIT(VIRTCHNL2_RXDID_19_COMMS_AUX_IPV6)
+#define VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW_M BIT(VIRTCHNL2_RXDID_20_COMMS_AUX_FLOW)
+#define VIRTCHNL2_RXDID_21_COMMS_AUX_TCP_M BIT(VIRTCHNL2_RXDID_21_COMMS_AUX_TCP)
+/* 22 through 63 are reserved */
+
+/* Rx */
+/* For splitq virtchnl2_rx_flex_desc_adv desc members */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_RXDID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_M \
+ MAKEMASK(0x3UL, VIRTCHNL2_RX_FLEX_DESC_ADV_UMBCAST_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_M \
+ MAKEMASK(0xFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF0_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_HDR_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_RSC_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_SPH_S)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M \
+ MAKEMASK(0x7UL, VIRTCHNL2_RX_FLEX_DESC_ADV_FF1_M)
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_M \
+ BIT_ULL(VIRTCHNL2_RX_FLEX_DESC_ADV_MISS_S)
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S 7
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_0_QW0_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LPBK_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_IPV6EXADD_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RXE_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_CRCP_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L2TAG1P_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD0_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XTRMD1_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_LAST 8 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS_ERROR_1_BITS
+ * for splitq virtchnl2_rx_flex_desc_adv
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_RSVD_S 0 /* 2 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_ATRAEFAIL_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_L2TAG2P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD2_VALID_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD3_VALID_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD4_VALID_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_XTRMD5_VALID_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS1_LAST 8 /* this entry must be last!!! */
+
+/* for singleq (flex) virtchnl2_rx_flex_desc fields */
+/* for virtchnl2_rx_flex_desc.ptype_flex_flags0 member */
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PTYPE_M \
+ MAKEMASK(0x3FFUL, VIRTCHNL2_RX_FLEX_DESC_PTYPE_S) /* 10 bits */
+
+/* for virtchnl2_rx_flex_desc.pkt_length member */
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M \
+ MAKEMASK(0x3FFFUL, VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_S) /* 14 bits */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_0_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S 0
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_EOF_S 1
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_HBO_S 2
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S 3
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S 5
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S 6
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S 7
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LPBK_S 8
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_IPV6EXADD_S 9
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RXE_S 10
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_CRCP_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_RSS_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_L2TAG1P_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS0_LAST 16 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_FLEX_DESC_STATUS_ERROR_1_BITS
+ * for singleq (flex) virtchnl2_rx_flex_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CPM_S 0 /* 4 bits */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_NAT_S 4
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_CRYPTO_S 5
+/* [10:6] reserved */
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_L2TAG2P_S 11
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD2_VALID_S 12
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD3_VALID_S 13
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD4_VALID_S 14
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_XTRMD5_VALID_S 15
+#define VIRTCHNL2_RX_FLEX_DESC_STATUS1_LAST 16 /* this entry must be last!!! */
+
+/* for virtchnl2_rx_flex_desc.ts_low member */
+#define VIRTCHNL2_RX_FLEX_TSTAMP_VALID BIT(0)
+
+/* For singleq (non flex) virtchnl2_singleq_base_rx_desc legacy desc members */
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S 63
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_M \
+ BIT_ULL(VIRTCHNL2_RX_BASE_DESC_QW1_LEN_SPH_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S 52
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_M \
+ MAKEMASK(0x7FFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_HBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S 38
+#define VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_M \
+ MAKEMASK(0x3FFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_LEN_PBUF_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S 30
+#define VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_M \
+ MAKEMASK(0xFFULL, VIRTCHNL2_RX_BASE_DESC_QW1_PTYPE_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S 19
+#define VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_M \
+ MAKEMASK(0xFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_ERROR_S)
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S 0
+#define VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_M \
+ MAKEMASK(0x7FFFFUL, VIRTCHNL2_RX_BASE_DESC_QW1_STATUS_S)
+
+/* VIRTCHNL2_RX_BASE_DESC_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_DD_S 0
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EOF_S 1
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L2TAG1P_S 2
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_L3L4P_S 3
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_CRCP_S 4
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD_S 5 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_EXT_UDP_0_S 8
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_UMBCAST_S 9 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLM_S 11
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_FLTSTAT_S 12 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LPBK_S 14
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_IPV6EXADD_S 15
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_RSVD1_S 16 /* 2 bits */
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_INT_UDP_0_S 18
+#define VIRTCHNL2_RX_BASE_DESC_STATUS_LAST 19 /* this entry must be last!!! */
+
+/* VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_EXT_STATUS_L2TAG2P_S 0
+
+/* VIRTCHNL2_RX_BASE_DESC_ERROR_BITS
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_RXE_S 0
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_ATRAEFAIL_S 1
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_HBO_S 2
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L3L4E_S 3 /* 3 bits */
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_IPE_S 3
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_L4E_S 4
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_EIPE_S 5
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_OVERSIZE_S 6
+#define VIRTCHNL2_RX_BASE_DESC_ERROR_PPRS_S 7
+
+/* VIRTCHNL2_RX_BASE_DESC_FLTSTAT_VALUES
+ * for singleq (base) virtchnl2_rx_base_desc
+ * Note: These are predefined bit offsets
+ */
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_NO_DATA 0
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_FD_ID 1
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSV 2
+#define VIRTCHNL2_RX_BASE_DESC_FLTSTAT_RSS_HASH 3
+
+/* Receive Descriptors */
+/* splitq buf
+ * | 16| 0|
+ * ----------------------------------------------------------------
+ * | RSV | Buffer ID |
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_splitq_rx_buf_desc {
+ struct {
+ __le16 buf_id; /* Buffer Identifier */
+ __le16 rsvd0;
+ __le32 rsvd1;
+ } qword0;
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+/* singleq buf
+ * | 0|
+ * ----------------------------------------------------------------
+ * | Rx packet buffer address |
+ * ----------------------------------------------------------------
+ * | Rx header buffer address |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | RSV |
+ * ----------------------------------------------------------------
+ * | 0|
+ */
+struct virtchnl2_singleq_rx_buf_desc {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ __le64 rsvd1;
+ __le64 rsvd2;
+}; /* read used with buffer queues*/
+
+union virtchnl2_rx_buf_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_splitq_rx_buf_desc split_rd;
+};
+
+/* (0x00) singleq wb(compl) */
+struct virtchnl2_singleq_base_rx_desc {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_ptype_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+}; /* writeback */
+
+/* (0x01) singleq flex compl */
+struct virtchnl2_rx_flex_desc {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* (0x02) */
+struct virtchnl2_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Switch Profile
+ * RxDID Profile Id 3
+ * Flex-field 0: Source Vsi
+ */
+struct virtchnl2_rx_flex_desc_sw {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 src_vsi; /* [10:15] are reserved */
+ __le16 flex_md1_rsvd;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 rsvd; /* flex words 2-3 are reserved */
+ __le32 ts_high;
+};
+
+
+/* Rx Flex Descriptor NIC Profile
+ * RxDID Profile Id 6
+ * Flex-field 0: RSS hash lower 16-bits
+ * Flex-field 1: RSS hash upper 16-bits
+ * Flex-field 2: Flow Id lower 16-bits
+ * Flex-field 3: Source Vsi
+ * Flex-field 4: reserved, Vlan id taken from L2Tag
+ */
+struct virtchnl2_rx_flex_desc_nic_2 {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flex_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flow_id;
+ __le16 src_vsi;
+ union {
+ struct {
+ __le16 rsvd;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Rx Flex Descriptor Advanced (Split Queue Model)
+ * RxDID Profile Id 7
+ */
+struct virtchnl2_rx_flex_desc_adv {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 fmd0;
+ __le16 fmd1;
+ /* Qword 2 */
+ __le16 fmd2;
+ u8 fflags2;
+ u8 hash3;
+ __le16 fmd3;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 fmd5;
+ __le16 fmd6;
+ __le16 fmd7_0;
+ __le16 fmd7_1;
+}; /* writeback */
+
+/* Rx Flex Descriptor Advanced (Split Queue Model) NIC Profile
+ * RxDID Profile Id 8
+ * Flex-field 0: BufferID
+ * Flex-field 1: Raw checksum/L2TAG1/RSC Seg Len (determined by HW)
+ * Flex-field 2: Hash[15:0]
+ * Flex-flags 2: Hash[23:16]
+ * Flex-field 3: L2TAG2
+ * Flex-field 5: L2TAG1
+ * Flex-field 7: Timestamp (upper 32 bits)
+ */
+struct virtchnl2_rx_flex_desc_adv_nic_3 {
+ /* Qword 0 */
+ u8 rxdid_ucast; /* profile_id=[3:0] */
+ /* rsvd=[5:4] */
+ /* ucast=[7:6] */
+ u8 status_err0_qw0;
+ __le16 ptype_err_fflags0; /* ptype=[9:0] */
+ /* ip_hdr_err=[10:10] */
+ /* udp_len_err=[11:11] */
+ /* ff0=[15:12] */
+ __le16 pktlen_gen_bufq_id; /* plen=[13:0] */
+ /* gen=[14:14] only in splitq */
+ /* bufq_id=[15:15] only in splitq */
+ __le16 hdrlen_flags; /* header=[9:0] */
+ /* rsc=[10:10] only in splitq */
+ /* sph=[11:11] only in splitq */
+ /* ext_udp_0=[12:12] */
+ /* int_udp_0=[13:13] */
+ /* trunc_mirr=[14:14] */
+ /* miss_prepend=[15:15] */
+ /* Qword 1 */
+ u8 status_err0_qw1;
+ u8 status_err1;
+ u8 fflags1;
+ u8 ts_low;
+ __le16 buf_id; /* only in splitq */
+ union {
+ __le16 raw_cs;
+ __le16 l2tag1;
+ __le16 rscseglen;
+ } misc;
+ /* Qword 2 */
+ __le16 hash1;
+ union {
+ u8 fflags2;
+ u8 mirrorid;
+ u8 hash2;
+ } ff2_mirrid_hash2;
+ u8 hash3;
+ __le16 l2tag2;
+ __le16 fmd4;
+ /* Qword 3 */
+ __le16 l2tag1;
+ __le16 fmd6;
+ __le32 ts_high;
+}; /* writeback */
+
+union virtchnl2_rx_desc {
+ struct virtchnl2_singleq_rx_buf_desc read;
+ struct virtchnl2_singleq_base_rx_desc base_wb;
+ struct virtchnl2_rx_flex_desc flex_wb;
+ struct virtchnl2_rx_flex_desc_nic flex_nic_wb;
+ struct virtchnl2_rx_flex_desc_sw flex_sw_wb;
+ struct virtchnl2_rx_flex_desc_nic_2 flex_nic_2_wb;
+ struct virtchnl2_rx_flex_desc_adv flex_adv_wb;
+ struct virtchnl2_rx_flex_desc_adv_nic_3 flex_adv_nic_3_wb;
+};
+
+#endif /* _VIRTCHNL_LAN_DESC_H_ */
diff --git a/drivers/common/idpf/base/virtchnl_inline_ipsec.h b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
new file mode 100644
index 0000000000..e19043ac47
--- /dev/null
+++ b/drivers/common/idpf/base/virtchnl_inline_ipsec.h
@@ -0,0 +1,567 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2001-2022 Intel Corporation
+ */
+
+#ifndef _VIRTCHNL_INLINE_IPSEC_H_
+#define _VIRTCHNL_INLINE_IPSEC_H_
+
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM 3
+#define VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM 16
+#define VIRTCHNL_IPSEC_MAX_TX_DESC_NUM 128
+#define VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER 2
+#define VIRTCHNL_IPSEC_MAX_KEY_LEN 128
+#define VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM 8
+#define VIRTCHNL_IPSEC_SA_DESTROY 0
+#define VIRTCHNL_IPSEC_BROADCAST_VFID 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_REQ_ID 0xFFFF
+#define VIRTCHNL_IPSEC_INVALID_SA_CFG_RESP 0xFFFFFFFF
+#define VIRTCHNL_IPSEC_INVALID_SP_CFG_RESP 0xFFFFFFFF
+
+/* crypto type */
+#define VIRTCHNL_AUTH 1
+#define VIRTCHNL_CIPHER 2
+#define VIRTCHNL_AEAD 3
+
+/* caps enabled */
+#define VIRTCHNL_IPSEC_ESN_ENA BIT(0)
+#define VIRTCHNL_IPSEC_UDP_ENCAP_ENA BIT(1)
+#define VIRTCHNL_IPSEC_SA_INDEX_SW_ENA BIT(2)
+#define VIRTCHNL_IPSEC_AUDIT_ENA BIT(3)
+#define VIRTCHNL_IPSEC_BYTE_LIMIT_ENA BIT(4)
+#define VIRTCHNL_IPSEC_DROP_ON_AUTH_FAIL_ENA BIT(5)
+#define VIRTCHNL_IPSEC_ARW_CHECK_ENA BIT(6)
+#define VIRTCHNL_IPSEC_24BIT_SPI_ENA BIT(7)
+
+/* algorithm type */
+/* Hash Algorithm */
+#define VIRTCHNL_HASH_NO_ALG 0 /* NULL algorithm */
+#define VIRTCHNL_AES_CBC_MAC 1 /* AES-CBC-MAC algorithm */
+#define VIRTCHNL_AES_CMAC 2 /* AES CMAC algorithm */
+#define VIRTCHNL_AES_GMAC 3 /* AES GMAC algorithm */
+#define VIRTCHNL_AES_XCBC_MAC 4 /* AES XCBC algorithm */
+#define VIRTCHNL_MD5_HMAC 5 /* HMAC using MD5 algorithm */
+#define VIRTCHNL_SHA1_HMAC 6 /* HMAC using 128 bit SHA algorithm */
+#define VIRTCHNL_SHA224_HMAC 7 /* HMAC using 224 bit SHA algorithm */
+#define VIRTCHNL_SHA256_HMAC 8 /* HMAC using 256 bit SHA algorithm */
+#define VIRTCHNL_SHA384_HMAC 9 /* HMAC using 384 bit SHA algorithm */
+#define VIRTCHNL_SHA512_HMAC 10 /* HMAC using 512 bit SHA algorithm */
+#define VIRTCHNL_SHA3_224_HMAC 11 /* HMAC using 224 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_256_HMAC 12 /* HMAC using 256 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_384_HMAC 13 /* HMAC using 384 bit SHA3 algorithm */
+#define VIRTCHNL_SHA3_512_HMAC 14 /* HMAC using 512 bit SHA3 algorithm */
+/* Cipher Algorithm */
+#define VIRTCHNL_CIPHER_NO_ALG 15 /* NULL algorithm */
+#define VIRTCHNL_3DES_CBC 16 /* Triple DES algorithm in CBC mode */
+#define VIRTCHNL_AES_CBC 17 /* AES algorithm in CBC mode */
+#define VIRTCHNL_AES_CTR 18 /* AES algorithm in Counter mode */
+/* AEAD Algorithm */
+#define VIRTCHNL_AES_CCM 19 /* AES algorithm in CCM mode */
+#define VIRTCHNL_AES_GCM 20 /* AES algorithm in GCM mode */
+#define VIRTCHNL_CHACHA20_POLY1305 21 /* algorithm of ChaCha20-Poly1305 */
+
+/* protocol type */
+#define VIRTCHNL_PROTO_ESP 1
+#define VIRTCHNL_PROTO_AH 2
+#define VIRTCHNL_PROTO_RSVD1 3
+
+/* sa mode */
+#define VIRTCHNL_SA_MODE_TRANSPORT 1
+#define VIRTCHNL_SA_MODE_TUNNEL 2
+#define VIRTCHNL_SA_MODE_TRAN_TUN 3
+#define VIRTCHNL_SA_MODE_UNKNOWN 4
+
+/* sa direction */
+#define VIRTCHNL_DIR_INGRESS 1
+#define VIRTCHNL_DIR_EGRESS 2
+#define VIRTCHNL_DIR_INGRESS_EGRESS 3
+
+/* sa termination */
+#define VIRTCHNL_TERM_SOFTWARE 1
+#define VIRTCHNL_TERM_HARDWARE 2
+
+/* sa ip type */
+#define VIRTCHNL_IPV4 1
+#define VIRTCHNL_IPV6 2
+
+/* for virtchnl_ipsec_resp */
+enum inline_ipsec_resp {
+ INLINE_IPSEC_SUCCESS = 0,
+ INLINE_IPSEC_FAIL = -1,
+ INLINE_IPSEC_ERR_FIFO_FULL = -2,
+ INLINE_IPSEC_ERR_NOT_READY = -3,
+ INLINE_IPSEC_ERR_VF_DOWN = -4,
+ INLINE_IPSEC_ERR_INVALID_PARAMS = -5,
+ INLINE_IPSEC_ERR_NO_MEM = -6,
+};
+
+/* Detailed opcodes for DPDK and IPsec use */
+enum inline_ipsec_ops {
+ INLINE_IPSEC_OP_GET_CAP = 0,
+ INLINE_IPSEC_OP_GET_STATUS = 1,
+ INLINE_IPSEC_OP_SA_CREATE = 2,
+ INLINE_IPSEC_OP_SA_UPDATE = 3,
+ INLINE_IPSEC_OP_SA_DESTROY = 4,
+ INLINE_IPSEC_OP_SP_CREATE = 5,
+ INLINE_IPSEC_OP_SP_DESTROY = 6,
+ INLINE_IPSEC_OP_SA_READ = 7,
+ INLINE_IPSEC_OP_EVENT = 8,
+ INLINE_IPSEC_OP_RESP = 9,
+};
+
+#pragma pack(1)
+/* Not all valid, if certain field is invalid, set 1 for all bits */
+struct virtchnl_algo_cap {
+ u32 algo_type;
+
+ u16 block_size;
+
+ u16 min_key_size;
+ u16 max_key_size;
+ u16 inc_key_size;
+
+ u16 min_iv_size;
+ u16 max_iv_size;
+ u16 inc_iv_size;
+
+ u16 min_digest_size;
+ u16 max_digest_size;
+ u16 inc_digest_size;
+
+ u16 min_aad_size;
+ u16 max_aad_size;
+ u16 inc_aad_size;
+};
+#pragma pack()
+
+/* vf record the capability of crypto from the virtchnl */
+struct virtchnl_sym_crypto_cap {
+ u8 crypto_type;
+ u8 algo_cap_num;
+ struct virtchnl_algo_cap algo_cap_list[VIRTCHNL_IPSEC_MAX_ALGO_CAP_NUM];
+};
+
+/* VIRTCHNL_OP_GET_IPSEC_CAP
+ * VF pass virtchnl_ipsec_cap to PF
+ * and PF return capability of ipsec from virtchnl.
+ */
+#pragma pack(1)
+struct virtchnl_ipsec_cap {
+ /* max number of SA per VF */
+ u16 max_sa_num;
+
+ /* IPsec SA Protocol - value ref VIRTCHNL_PROTO_XXX */
+ u8 virtchnl_protocol_type;
+
+ /* IPsec SA Mode - value ref VIRTCHNL_SA_MODE_XXX */
+ u8 virtchnl_sa_mode;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 termination_mode;
+
+ /* number of supported crypto capability */
+ u8 crypto_cap_num;
+
+ /* descriptor ID */
+ u16 desc_id;
+
+ /* capabilities enabled - value ref VIRTCHNL_IPSEC_XXX_ENA */
+ u32 caps_enabled;
+
+ /* crypto capabilities */
+ struct virtchnl_sym_crypto_cap cap[VIRTCHNL_IPSEC_MAX_CRYPTO_CAP_NUM];
+};
+
+/* configuration of crypto function */
+struct virtchnl_ipsec_crypto_cfg_item {
+ u8 crypto_type;
+
+ u32 algo_type;
+
+ /* Length of valid IV data. */
+ u16 iv_len;
+
+ /* Length of digest */
+ u16 digest_len;
+
+ /* SA salt */
+ u32 salt;
+
+ /* The length of the symmetric key */
+ u16 key_len;
+
+ /* key data buffer */
+ u8 key_data[VIRTCHNL_IPSEC_MAX_KEY_LEN];
+};
+#pragma pack()
+
+struct virtchnl_ipsec_sym_crypto_cfg {
+ struct virtchnl_ipsec_crypto_cfg_item
+ items[VIRTCHNL_IPSEC_MAX_CRYPTO_ITEM_NUMBER];
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_CREATE
+ * VF send this SA configuration to PF using virtchnl;
+ * PF create SA as configuration and PF driver will return
+ * an unique index (sa_idx) for the created SA.
+ */
+struct virtchnl_ipsec_sa_cfg {
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* type of outer IP - IPv4/IPv6 */
+ u8 virtchnl_ip_type;
+
+ /* type of esn - !0:enable/0:disable */
+ u8 esn_enabled;
+
+ /* udp encap - !0:enable/0:disable */
+ u8 udp_encap_enabled;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* outer src ip address */
+ u8 src_addr[16];
+
+ /* outer dst ip address */
+ u8 dst_addr[16];
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* When enabled, sa_index must be valid */
+ u8 sa_index_en;
+
+ /* SA index when sa_index_en is true */
+ u32 sa_index;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When enabled, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-reply window check - enable/disable
+ * When enabled, arw_size must be valid.
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* no ip offload mode - enable/disable
+ * When enabled, ip type and address must not be valid.
+ */
+ u8 no_ip_offload_en;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* crypto configuration */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* VIRTCHNL_OP_IPSEC_SA_UPDATE
+ * VF send configuration of index of SA to PF
+ * PF will update SA according to configuration
+ */
+struct virtchnl_ipsec_sa_update {
+ u32 sa_index; /* SA to update */
+ u32 esn_hi; /* high 32 bits of esn */
+ u32 esn_low; /* low 32 bits of esn */
+};
+
+#pragma pack(1)
+/* VIRTCHNL_OP_IPSEC_SA_DESTROY
+ * VF send configuration of index of SA to PF
+ * PF will destroy SA according to configuration
+ * flag bitmap indicate all SA or just selected SA will
+ * be destroyed
+ */
+struct virtchnl_ipsec_sa_destroy {
+ /* All zero bitmap indicates all SA will be destroyed.
+ * Non-zero bitmap indicates the selected SA in
+ * array sa_index will be destroyed.
+ */
+ u8 flag;
+
+ /* selected SA index */
+ u32 sa_index[VIRTCHNL_IPSEC_MAX_SA_DESTROY_NUM];
+};
+
+/* VIRTCHNL_OP_IPSEC_SA_READ
+ * VF send this SA configuration to PF using virtchnl;
+ * PF read SA and will return configuration for the created SA.
+ */
+struct virtchnl_ipsec_sa_read {
+ /* SA valid - invalid/valid */
+ u8 valid;
+
+ /* SA active - inactive/active */
+ u8 active;
+
+ /* SA SN rollover - not_rollover/rollover */
+ u8 sn_rollover;
+
+ /* IPsec SA Protocol - AH/ESP */
+ u8 virtchnl_protocol_type;
+
+ /* termination mode - value ref VIRTCHNL_TERM_XXX */
+ u8 virtchnl_termination;
+
+ /* auditing mode - enable/disable */
+ u8 audit_en;
+
+ /* lifetime byte limit - enable/disable
+ * When set to limit, byte_limit_hard and byte_limit_soft
+ * must be valid.
+ */
+ u8 byte_limit_en;
+
+ /* hard byte limit count */
+ u64 byte_limit_hard;
+
+ /* soft byte limit count */
+ u64 byte_limit_soft;
+
+ /* drop on authentication failure - enable/disable */
+ u8 drop_on_auth_fail_en;
+
+ /* anti-replay window check - enable/disable
+ * When set to check, arw_size, arw_top, and arw must be valid
+ */
+ u8 arw_check_en;
+
+ /* size of arw window, offset by 1. Setting to 0
+ * represents ARW window size of 1. Setting to 127
+ * represents ARW window size of 128
+ */
+ u8 arw_size;
+
+ /* reserved */
+ u8 reserved1;
+
+ /* top of anti-replay-window */
+ u64 arw_top;
+
+ /* anti-replay-window */
+ u8 arw[16];
+
+ /* packets processed */
+ u64 packets_processed;
+
+ /* bytes processed */
+ u64 bytes_processed;
+
+ /* packets dropped */
+ u32 packets_dropped;
+
+ /* authentication failures */
+ u32 auth_fails;
+
+ /* ARW check failures */
+ u32 arw_fails;
+
+ /* type of esn - enable/disable */
+ u8 esn;
+
+ /* IPSec SA Direction - value ref VIRTCHNL_DIR_XXX */
+ u8 virtchnl_direction;
+
+ /* SA security parameter index */
+ u32 spi;
+
+ /* SA salt */
+ u32 salt;
+
+ /* high 32 bits of esn */
+ u32 esn_hi;
+
+ /* low 32 bits of esn */
+ u32 esn_low;
+
+ /* SA Domain. Used to logical separate an SADB into groups.
+ * PF drivers supporting a single group ignore this field.
+ */
+ u16 sa_domain;
+
+ /* SPD reference. Used to link an SA with its policy.
+ * PF drivers may ignore this field.
+ */
+ u16 spd_ref;
+
+ /* crypto configuration. Salt and keys are set to 0 */
+ struct virtchnl_ipsec_sym_crypto_cfg crypto_cfg;
+};
+#pragma pack()
+
+/* Add allowlist entry in IES */
+struct virtchnl_ipsec_sp_cfg {
+ u32 spi;
+ u32 dip[4];
+
+ /* Drop frame if true or redirect to QAT if false. */
+ u8 drop;
+
+ /* Congestion domain. For future use. */
+ u8 cgd;
+
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+
+ /* Set TC (congestion domain) if true. For future use. */
+ u8 set_tc;
+
+ /* 0 for NAT-T unsupported, 1 for NAT-T supported */
+ u8 is_udp;
+
+ /* reserved */
+ u8 reserved;
+
+ /* NAT-T UDP port number. Only valid in case NAT-T supported */
+ u16 udp_port;
+};
+
+#pragma pack(1)
+/* Delete allowlist entry in IES */
+struct virtchnl_ipsec_sp_destroy {
+ /* 0 for IPv4 table, 1 for IPv6 table. */
+ u8 table_id;
+ u32 rule_id;
+};
+#pragma pack()
+
+/* Response from IES to allowlist operations */
+struct virtchnl_ipsec_sp_cfg_resp {
+ u32 rule_id;
+};
+
+struct virtchnl_ipsec_sa_cfg_resp {
+ u32 sa_handle;
+};
+
+#define INLINE_IPSEC_EVENT_RESET 0x1
+#define INLINE_IPSEC_EVENT_CRYPTO_ON 0x2
+#define INLINE_IPSEC_EVENT_CRYPTO_OFF 0x4
+
+struct virtchnl_ipsec_event {
+ u32 ipsec_event_data;
+};
+
+#define INLINE_IPSEC_STATUS_AVAILABLE 0x1
+#define INLINE_IPSEC_STATUS_UNAVAILABLE 0x2
+
+struct virtchnl_ipsec_status {
+ u32 status;
+};
+
+struct virtchnl_ipsec_resp {
+ u32 resp;
+};
+
+/* Internal message descriptor for VF <-> IPsec communication */
+struct inline_ipsec_msg {
+ u16 ipsec_opcode;
+ u16 req_id;
+
+ union {
+ /* IPsec request */
+ struct virtchnl_ipsec_sa_cfg sa_cfg[0];
+ struct virtchnl_ipsec_sp_cfg sp_cfg[0];
+ struct virtchnl_ipsec_sa_update sa_update[0];
+ struct virtchnl_ipsec_sa_destroy sa_destroy[0];
+ struct virtchnl_ipsec_sp_destroy sp_destroy[0];
+
+ /* IPsec response */
+ struct virtchnl_ipsec_sa_cfg_resp sa_cfg_resp[0];
+ struct virtchnl_ipsec_sp_cfg_resp sp_cfg_resp[0];
+ struct virtchnl_ipsec_cap ipsec_cap[0];
+ struct virtchnl_ipsec_status ipsec_status[0];
+ /* response to del_sa, del_sp, update_sa */
+ struct virtchnl_ipsec_resp ipsec_resp[0];
+
+ /* IPsec event (no req_id is required) */
+ struct virtchnl_ipsec_event event[0];
+
+ /* Reserved */
+ struct virtchnl_ipsec_sa_read sa_read[0];
+ } ipsec_data;
+};
+
+static inline u16 virtchnl_inline_ipsec_val_msg_len(u16 opcode)
+{
+ u16 valid_len = sizeof(struct inline_ipsec_msg);
+
+ switch (opcode) {
+ case INLINE_IPSEC_OP_GET_CAP:
+ case INLINE_IPSEC_OP_GET_STATUS:
+ break;
+ case INLINE_IPSEC_OP_SA_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_cfg);
+ break;
+ case INLINE_IPSEC_OP_SP_CREATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_cfg);
+ break;
+ case INLINE_IPSEC_OP_SA_UPDATE:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_update);
+ break;
+ case INLINE_IPSEC_OP_SA_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sa_destroy);
+ break;
+ case INLINE_IPSEC_OP_SP_DESTROY:
+ valid_len += sizeof(struct virtchnl_ipsec_sp_destroy);
+ break;
+ /* Only for msg length calculation of response to VF in case of
+ * inline ipsec failure.
+ */
+ case INLINE_IPSEC_OP_RESP:
+ valid_len += sizeof(struct virtchnl_ipsec_resp);
+ break;
+ default:
+ valid_len = 0;
+ break;
+ }
+
+ return valid_len;
+}
+
+#endif /* _VIRTCHNL_INLINE_IPSEC_H_ */
diff --git a/drivers/common/idpf/meson.build b/drivers/common/idpf/meson.build
new file mode 100644
index 0000000000..26881166e9
--- /dev/null
+++ b/drivers/common/idpf/meson.build
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+subdir('base')
\ No newline at end of file
diff --git a/drivers/common/idpf/version.map b/drivers/common/idpf/version.map
new file mode 100644
index 0000000000..bfb246c752
--- /dev/null
+++ b/drivers/common/idpf/version.map
@@ -0,0 +1,12 @@
+INTERNAL {
+ global:
+
+ idpf_ctlq_deinit;
+ idpf_ctlq_init;
+ idpf_ctlq_clean_sq;
+ idpf_ctlq_recv;
+ idpf_ctlq_send;
+ idpf_ctlq_post_rx_buffs;
+
+ local: *;
+};
diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index ea261dd70a..b63d899d50 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -6,6 +6,7 @@ drivers = [
'cpt',
'dpaax',
'iavf',
+ 'idpf',
'mvep',
'octeontx',
]
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 02/18] net/idpf: add support for device initialization
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 8:33 ` [PATCH v18 01/18] common/idpf: introduce common library beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 03/18] net/idpf: add Tx queue setup beilei.xing
` (16 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing
Cc: dev, Junfeng Guo, Xiaoyun Li, Xiao Wang, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Support device init and add the following dev ops:
- dev_configure
- dev_close
- dev_infos_get
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
MAINTAINERS | 9 +
doc/guides/nics/features/idpf.ini | 9 +
doc/guides/nics/idpf.rst | 66 ++
doc/guides/nics/index.rst | 1 +
doc/guides/rel_notes/release_22_11.rst | 6 +
drivers/net/idpf/idpf_ethdev.c | 891 +++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 189 ++++++
drivers/net/idpf/idpf_logs.h | 56 ++
drivers/net/idpf/idpf_vchnl.c | 416 ++++++++++++
drivers/net/idpf/meson.build | 15 +
drivers/net/idpf/version.map | 3 +
drivers/net/meson.build | 1 +
12 files changed, 1662 insertions(+)
create mode 100644 doc/guides/nics/features/idpf.ini
create mode 100644 doc/guides/nics/idpf.rst
create mode 100644 drivers/net/idpf/idpf_ethdev.c
create mode 100644 drivers/net/idpf/idpf_ethdev.h
create mode 100644 drivers/net/idpf/idpf_logs.h
create mode 100644 drivers/net/idpf/idpf_vchnl.c
create mode 100644 drivers/net/idpf/meson.build
create mode 100644 drivers/net/idpf/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index bdf233c9f8..cc66db25e8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -770,6 +770,15 @@ F: drivers/net/ice/
F: doc/guides/nics/ice.rst
F: doc/guides/nics/features/ice.ini
+Intel idpf
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Beilei Xing <beilei.xing@intel.com>
+T: git://dpdk.org/next/dpdk-next-net-intel
+F: drivers/net/idpf/
+F: drivers/common/idpf/
+F: doc/guides/nics/idpf.rst
+F: doc/guides/nics/features/idpf.ini
+
Intel igc
M: Junfeng Guo <junfeng.guo@intel.com>
M: Simei Su <simei.su@intel.com>
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
new file mode 100644
index 0000000000..46aab2eb61
--- /dev/null
+++ b/doc/guides/nics/features/idpf.ini
@@ -0,0 +1,9 @@
+;
+; Supported features of the 'idpf' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
new file mode 100644
index 0000000000..c1001d5d0c
--- /dev/null
+++ b/doc/guides/nics/idpf.rst
@@ -0,0 +1,66 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 Intel Corporation.
+
+IDPF Poll Mode Driver
+======================
+
+The [*EXPERIMENTAL*] idpf PMD (**librte_net_idpf**) provides poll mode driver support for
+Intel® Infrastructure Processing Unit (Intel® IPU) E2000.
+
+
+Linux Prerequisites
+-------------------
+
+- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
+
+- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
+ section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
+
+
+Pre-Installation Configuration
+------------------------------
+
+Runtime Config Options
+~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vport`` (default ``0``)
+
+ The IDPF PMD supports creation of multiple vports for one PCI device, each vport
+ corresponds to a single ethdev. Using the ``devargs`` parameter ``vport`` the user
+ can specify the vports with specific ID to be created, for example::
+
+ -a ca:00.0,vport=[0,2,3]
+
+ Then idpf PMD will create 3 vports (ethdevs) for device ca:00.0.
+ NOTE: If the parameter is not provided, the vport 0 will be created by default.
+
+- ``rx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Rx queue. User can choose Rx queue mode by the ``devargs``
+ parameter ``rx_single``.
+
+ -a ca:00.0,rx_single=1
+
+ Then idpf PMD will configure Rx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+- ``tx_single`` (default ``0``)
+
+ There're two queue modes supported by Intel® IPU Ethernet ES2000 Series, single queue
+ mode and split queue mode for Tx queue. User can choose Tx queue mode by the ``devargs``
+ parameter ``tx_single``.
+
+ -a ca:00.0,tx_single=1
+
+ Then idpf PMD will configure Tx queue with single queue mode. Otherwise, split queue
+ mode is chosen by default.
+
+
+Driver compilation and testing
+------------------------------
+
+Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
+for details.
+
+
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 4d40ea29a3..12841ce407 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -34,6 +34,7 @@ Network Interface Controller Drivers
hns3
i40e
ice
+ idpf
igb
igc
ionic
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 28812e092f..77674f2b06 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -158,6 +158,12 @@ New Features
* Added protocol based buffer split support in scalar path.
+* **Added IDPF PMD [*EXPERIMENTAL*].**
+
+ Added the new ``idpf`` net driver for Intel® Infrastructure Processing Unit
+ (Intel® IPU) E2000.
+ See the :doc:`../nics/idpf` NIC guide for more details on this new driver.
+
* **Updated Marvell cnxk driver.**
* Added support for flow action REPRESENTED_PORT.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
new file mode 100644
index 0000000000..035f563275
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -0,0 +1,891 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <rte_dev.h>
+#include <errno.h>
+
+#include "idpf_ethdev.h"
+
+#define IDPF_TX_SINGLE_Q "tx_single"
+#define IDPF_RX_SINGLE_Q "rx_single"
+#define IDPF_VPORT "vport"
+
+rte_spinlock_t idpf_adapter_lock;
+/* A list for all adapters, one adapter matches one PCI device */
+struct idpf_adapter_list idpf_adapter_list;
+bool idpf_adapter_list_init;
+
+static const char * const idpf_valid_args[] = {
+ IDPF_TX_SINGLE_Q,
+ IDPF_RX_SINGLE_Q,
+ IDPF_VPORT,
+ NULL
+};
+
+static int
+idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ dev_info->max_rx_queues = adapter->caps->max_rx_q;
+ dev_info->max_tx_queues = adapter->caps->max_tx_q;
+ dev_info->min_rx_bufsize = IDPF_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = IDPF_MAX_FRAME_SIZE;
+
+ dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ return 0;
+}
+
+static int
+idpf_init_vport_req_info(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_create_vport *vport_info;
+ uint16_t idx = adapter->cur_vport_idx;
+
+ if (idx == IDPF_INVALID_VPORT_IDX) {
+ PMD_INIT_LOG(ERR, "Invalid vport index.");
+ return -EINVAL;
+ }
+
+ if (adapter->vport_req_info[idx] == NULL) {
+ adapter->vport_req_info[idx] = rte_zmalloc(NULL,
+ sizeof(struct virtchnl2_create_vport), 0);
+ if (adapter->vport_req_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info");
+ return -ENOMEM;
+ }
+ }
+
+ vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+
+ vport_info->vport_type = rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
+ if (adapter->txq_model == 0) {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq =
+ IDPF_DEFAULT_TXQ_NUM * IDPF_TX_COMPLQ_PER_GRP;
+ } else {
+ vport_info->txq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_tx_q = IDPF_DEFAULT_TXQ_NUM;
+ vport_info->num_tx_complq = 0;
+ }
+ if (adapter->rxq_model == 0) {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SPLIT);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq =
+ IDPF_DEFAULT_RXQ_NUM * IDPF_RX_BUFQ_PER_GRP;
+ } else {
+ vport_info->rxq_model =
+ rte_cpu_to_le_16(VIRTCHNL2_QUEUE_MODEL_SINGLE);
+ vport_info->num_rx_q = IDPF_DEFAULT_RXQ_NUM;
+ vport_info->num_rx_bufq = 0;
+ }
+
+ return 0;
+}
+
+static int
+idpf_parse_devarg_id(char *name)
+{
+ uint16_t val;
+ char *p;
+
+ p = strstr(name, "vport_");
+
+ if (p == NULL)
+ return -EINVAL;
+
+ p += sizeof("vport_") - 1;
+
+ val = strtoul(p, NULL, 10);
+
+ return val;
+}
+
+static int
+idpf_init_vport(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_info =
+ (struct virtchnl2_create_vport *)adapter->vport_recv_info[idx];
+ int i, type, ret;
+
+ vport->vport_id = vport_info->vport_id;
+ vport->txq_model = vport_info->txq_model;
+ vport->rxq_model = vport_info->rxq_model;
+ vport->num_tx_q = vport_info->num_tx_q;
+ vport->num_tx_complq = vport_info->num_tx_complq;
+ vport->num_rx_q = vport_info->num_rx_q;
+ vport->num_rx_bufq = vport_info->num_rx_bufq;
+ vport->max_mtu = vport_info->max_mtu;
+ rte_memcpy(vport->default_mac_addr,
+ vport_info->default_mac_addr, ETH_ALEN);
+ vport->sw_idx = idx;
+
+ for (i = 0; i < vport_info->chunks.num_chunks; i++) {
+ type = vport_info->chunks.chunks[i].type;
+ switch (type) {
+ case VIRTCHNL2_QUEUE_TYPE_TX:
+ vport->chunks_info.tx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX:
+ vport->chunks_info.rx_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
+ vport->chunks_info.tx_compl_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.tx_compl_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.tx_compl_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
+ vport->chunks_info.rx_buf_start_qid =
+ vport_info->chunks.chunks[i].start_queue_id;
+ vport->chunks_info.rx_buf_qtail_start =
+ vport_info->chunks.chunks[i].qtail_reg_start;
+ vport->chunks_info.rx_buf_qtail_spacing =
+ vport_info->chunks.chunks[i].qtail_reg_spacing;
+ break;
+ default:
+ PMD_INIT_LOG(ERR, "Unsupported queue type");
+ break;
+ }
+ }
+
+ ret = idpf_parse_devarg_id(dev->data->name);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse devarg id.");
+ return -EINVAL;
+ }
+ vport->devarg_id = ret;
+
+ vport->dev_data = dev->data;
+
+ adapter->vports[idx] = vport;
+
+ return 0;
+}
+
+static int
+idpf_dev_configure(struct rte_eth_dev *dev)
+{
+ struct rte_eth_conf *conf = &dev->data->dev_conf;
+
+ if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
+ PMD_INIT_LOG(ERR, "Setting link speed is not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
+ conf->rxmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->txmode.mq_mode != RTE_ETH_MQ_TX_NONE) {
+ PMD_INIT_LOG(ERR, "Multi-queue TX mode %d is not supported",
+ conf->txmode.mq_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->lpbk_mode != 0) {
+ PMD_INIT_LOG(ERR, "Loopback operation mode %d is not supported",
+ conf->lpbk_mode);
+ return -ENOTSUP;
+ }
+
+ if (conf->dcb_capability_en != 0) {
+ PMD_INIT_LOG(ERR, "Priority Flow Control(PFC) if not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.lsc != 0) {
+ PMD_INIT_LOG(ERR, "LSC interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rxq != 0) {
+ PMD_INIT_LOG(ERR, "RXQ interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ if (conf->intr_conf.rmv != 0) {
+ PMD_INIT_LOG(ERR, "RMV interrupt is not supported");
+ return -ENOTSUP;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_close(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+
+ idpf_vc_destroy_vport(vport);
+
+
+ adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
+
+ rte_free(vport);
+ dev->data->dev_private = NULL;
+
+ return 0;
+}
+
+static int
+insert_value(struct idpf_adapter *adapter, uint16_t id)
+{
+ uint16_t i;
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if (adapter->req_vports[i] == id)
+ return 0;
+ }
+
+ if (adapter->req_vport_nb >= RTE_DIM(adapter->req_vports)) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ adapter->req_vports[adapter->req_vport_nb] = id;
+ adapter->req_vport_nb++;
+
+ return 0;
+}
+
+static const char *
+parse_range(const char *value, struct idpf_adapter *adapter)
+{
+ uint16_t lo, hi, i;
+ int n = 0;
+ int result;
+ const char *pos = value;
+
+ result = sscanf(value, "%hu%n-%hu%n", &lo, &n, &hi, &n);
+ if (result == 1) {
+ if (lo >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ if (insert_value(adapter, lo) != 0)
+ return NULL;
+ } else if (result == 2) {
+ if (lo > hi || hi >= IDPF_MAX_VPORT_NUM)
+ return NULL;
+ for (i = lo; i <= hi; i++) {
+ if (insert_value(adapter, i) != 0)
+ return NULL;
+ }
+ } else {
+ return NULL;
+ }
+
+ return pos + n;
+}
+
+static int
+parse_vport(const char *key, const char *value, void *args)
+{
+ struct idpf_adapter *adapter = args;
+ const char *pos = value;
+ int i;
+
+ adapter->req_vport_nb = 0;
+
+ if (*pos == '[')
+ pos++;
+
+ while (1) {
+ pos = parse_range(pos, adapter);
+ if (pos == NULL) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+ if (*pos != ',')
+ break;
+ pos++;
+ }
+
+ if (*value == '[' && *pos != ']') {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", ",
+ value, key);
+ return -EINVAL;
+ }
+
+ if (adapter->cur_vport_nb + adapter->req_vport_nb >
+ IDPF_MAX_VPORT_NUM) {
+ PMD_INIT_LOG(ERR, "Total vport number can't be > %d",
+ IDPF_MAX_VPORT_NUM);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ if ((adapter->cur_vports & RTE_BIT32(adapter->req_vports[i])) == 0) {
+ adapter->cur_vports |= RTE_BIT32(adapter->req_vports[i]);
+ adapter->cur_vport_nb++;
+ } else {
+ PMD_INIT_LOG(ERR, "Vport %d has been created",
+ adapter->req_vports[i]);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int
+parse_bool(const char *key, const char *value, void *args)
+{
+ int *i = args;
+ char *end;
+ int num;
+
+ errno = 0;
+
+ num = strtoul(value, &end, 10);
+
+ if (errno == ERANGE || (num != 0 && num != 1)) {
+ PMD_INIT_LOG(ERR, "invalid value:\"%s\" for key:\"%s\", value must be 0 or 1",
+ value, key);
+ return -EINVAL;
+ }
+
+ *i = num;
+ return 0;
+}
+
+static int
+idpf_parse_devargs(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct rte_devargs *devargs = pci_dev->device.devargs;
+ struct rte_kvargs *kvlist;
+ int ret;
+
+ if (devargs == NULL)
+ return 0;
+
+ kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
+ if (kvlist == NULL) {
+ PMD_INIT_LOG(ERR, "invalid kvargs key");
+ return -EINVAL;
+ }
+
+ ret = rte_kvargs_process(kvlist, IDPF_VPORT, &parse_vport,
+ adapter);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_TX_SINGLE_Q, &parse_bool,
+ &adapter->txq_model);
+ if (ret != 0)
+ goto bail;
+
+ ret = rte_kvargs_process(kvlist, IDPF_RX_SINGLE_Q, &parse_bool,
+ &adapter->rxq_model);
+ if (ret != 0)
+ goto bail;
+
+bail:
+ rte_kvargs_free(kvlist);
+ return ret;
+}
+
+static void
+idpf_reset_pf(struct idpf_hw *hw)
+{
+ uint32_t reg;
+
+ reg = IDPF_READ_REG(hw, PFGEN_CTRL);
+ IDPF_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR));
+}
+
+#define IDPF_RESET_WAIT_CNT 100
+static int
+idpf_check_pf_reset_done(struct idpf_hw *hw)
+{
+ uint32_t reg;
+ int i;
+
+ for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
+ reg = IDPF_READ_REG(hw, PFGEN_RSTAT);
+ if (reg != 0xFFFFFFFF && (reg & PFGEN_RSTAT_PFR_STATE_M))
+ return 0;
+ rte_delay_ms(1000);
+ }
+
+ PMD_INIT_LOG(ERR, "IDPF reset timeout");
+ return -EBUSY;
+}
+
+#define CTLQ_NUM 2
+static int
+idpf_init_mbx(struct idpf_hw *hw)
+{
+ struct idpf_ctlq_create_info ctlq_info[CTLQ_NUM] = {
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_TX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ATQH,
+ .tail = PF_FW_ATQT,
+ .len = PF_FW_ATQLEN,
+ .bah = PF_FW_ATQBAH,
+ .bal = PF_FW_ATQBAL,
+ .len_mask = PF_FW_ATQLEN_ATQLEN_M,
+ .len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M,
+ .head_mask = PF_FW_ATQH_ATQH_M,
+ }
+ },
+ {
+ .type = IDPF_CTLQ_TYPE_MAILBOX_RX,
+ .id = IDPF_CTLQ_ID,
+ .len = IDPF_CTLQ_LEN,
+ .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
+ .reg = {
+ .head = PF_FW_ARQH,
+ .tail = PF_FW_ARQT,
+ .len = PF_FW_ARQLEN,
+ .bah = PF_FW_ARQBAH,
+ .bal = PF_FW_ARQBAL,
+ .len_mask = PF_FW_ARQLEN_ARQLEN_M,
+ .len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M,
+ .head_mask = PF_FW_ARQH_ARQH_M,
+ }
+ }
+ };
+ struct idpf_ctlq_info *ctlq;
+ int ret;
+
+ ret = idpf_ctlq_init(hw, CTLQ_NUM, ctlq_info);
+ if (ret != 0)
+ return ret;
+
+ LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
+ struct idpf_ctlq_info, cq_list) {
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_TX)
+ hw->asq = ctlq;
+ if (ctlq->q_id == IDPF_CTLQ_ID &&
+ ctlq->cq_type == IDPF_CTLQ_TYPE_MAILBOX_RX)
+ hw->arq = ctlq;
+ }
+
+ if (hw->asq == NULL || hw->arq == NULL) {
+ idpf_ctlq_deinit(hw);
+ ret = -ENOENT;
+ }
+
+ return ret;
+}
+
+static int
+idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int ret = 0;
+
+ hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
+ hw->hw_addr_len = pci_dev->mem_resource[0].len;
+ hw->back = adapter;
+ hw->vendor_id = pci_dev->id.vendor_id;
+ hw->device_id = pci_dev->id.device_id;
+ hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
+
+ strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
+
+ idpf_reset_pf(hw);
+ ret = idpf_check_pf_reset_done(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "IDPF is still resetting");
+ goto err;
+ }
+
+ ret = idpf_init_mbx(hw);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init mailbox");
+ goto err;
+ }
+
+ adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->mbx_resp == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_adapter_mbx_resp memory");
+ ret = -ENOMEM;
+ goto err_mbx;
+ }
+
+ ret = idpf_vc_check_api_version(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to check api version");
+ goto err_api;
+ }
+
+ adapter->caps = rte_zmalloc("idpf_caps",
+ sizeof(struct virtchnl2_get_capabilities), 0);
+ if (adapter->caps == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
+ ret = -ENOMEM;
+ goto err_api;
+ }
+
+ ret = idpf_vc_get_caps(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to get capabilities");
+ goto err_caps;
+ }
+
+ adapter->max_vport_nb = adapter->caps->max_vports;
+
+ adapter->vport_req_info = rte_zmalloc("vport_req_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_req_info),
+ 0);
+ if (adapter->vport_req_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info memory");
+ ret = -ENOMEM;
+ goto err_caps;
+ }
+
+ adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vport_recv_info),
+ 0);
+ if (adapter->vport_recv_info == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info memory");
+ ret = -ENOMEM;
+ goto err_vport_recv_info;
+ }
+
+ adapter->vports = rte_zmalloc("vports",
+ adapter->max_vport_nb *
+ sizeof(*adapter->vports),
+ 0);
+ if (adapter->vports == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
+ ret = -ENOMEM;
+ goto err_vports;
+ }
+
+ adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_rx_queues)) /
+ sizeof(struct virtchnl2_rxq_info);
+ adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
+ sizeof(struct virtchnl2_config_tx_queues)) /
+ sizeof(struct virtchnl2_txq_info);
+
+ adapter->cur_vports = 0;
+ adapter->cur_vport_nb = 0;
+
+ return ret;
+
+err_vports:
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+err_vport_recv_info:
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+err_caps:
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+err_api:
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+err_mbx:
+ idpf_ctlq_deinit(hw);
+err:
+ return ret;
+}
+
+static const struct eth_dev_ops idpf_eth_dev_ops = {
+ .dev_configure = idpf_dev_configure,
+ .dev_close = idpf_dev_close,
+ .dev_infos_get = idpf_dev_info_get,
+};
+
+static uint16_t
+idpf_get_vport_idx(struct idpf_vport **vports, uint16_t max_vport_nb)
+{
+ uint16_t vport_idx;
+ uint16_t i;
+
+ for (i = 0; i < max_vport_nb; i++) {
+ if (vports[i] == NULL)
+ break;
+ }
+
+ if (i == max_vport_nb)
+ vport_idx = IDPF_INVALID_VPORT_IDX;
+ else
+ vport_idx = i;
+
+ return vport_idx;
+}
+
+static int
+idpf_dev_init(struct rte_eth_dev *dev, void *init_params)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = init_params;
+ int ret = 0;
+
+ dev->dev_ops = &idpf_eth_dev_ops;
+ vport->adapter = adapter;
+
+ ret = idpf_init_vport_req_info(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
+ goto err;
+ }
+
+ ret = idpf_vc_create_vport(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to create vport.");
+ goto err_create_vport;
+ }
+
+ ret = idpf_init_vport(dev);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init vports.");
+ goto err_init_vport;
+ }
+
+ adapter->cur_vport_idx = idpf_get_vport_idx(adapter->vports,
+ adapter->max_vport_nb);
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
+ ret = -ENOMEM;
+ goto err_init_vport;
+ }
+
+ rte_ether_addr_copy((struct rte_ether_addr *)vport->default_mac_addr,
+ &dev->data->mac_addrs[0]);
+
+ return 0;
+
+err_init_vport:
+ idpf_vc_destroy_vport(vport);
+err_create_vport:
+ rte_free(vport->adapter->vport_req_info[vport->adapter->cur_vport_idx]);
+err:
+ return ret;
+}
+
+static const struct rte_pci_id pci_id_idpf_map[] = {
+ { RTE_PCI_DEVICE(IDPF_INTEL_VENDOR_ID, IDPF_DEV_ID_PF) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+struct idpf_adapter *
+idpf_find_adapter(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ int found = 0;
+
+ if (pci_dev == NULL)
+ return NULL;
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_FOREACH(adapter, &idpf_adapter_list, next) {
+ if (strncmp(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE) == 0) {
+ found = 1;
+ break;
+ }
+ }
+ rte_spinlock_unlock(&idpf_adapter_lock);
+
+ if (found == 0)
+ return NULL;
+
+ return adapter;
+}
+
+static void
+idpf_adapter_rel(struct idpf_adapter *adapter)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ int i;
+
+ idpf_ctlq_deinit(hw);
+
+ rte_free(adapter->caps);
+ adapter->caps = NULL;
+
+ rte_free(adapter->mbx_resp);
+ adapter->mbx_resp = NULL;
+
+ if (adapter->vport_req_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_req_info[i]);
+ adapter->vport_req_info[i] = NULL;
+ }
+ rte_free(adapter->vport_req_info);
+ adapter->vport_req_info = NULL;
+ }
+
+ if (adapter->vport_recv_info != NULL) {
+ for (i = 0; i < adapter->max_vport_nb; i++) {
+ rte_free(adapter->vport_recv_info[i]);
+ adapter->vport_recv_info[i] = NULL;
+ }
+ rte_free(adapter->vport_recv_info);
+ adapter->vport_recv_info = NULL;
+ }
+
+ rte_free(adapter->vports);
+ adapter->vports = NULL;
+}
+
+static int
+idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter;
+ char name[RTE_ETH_NAME_MAX_LEN];
+ int i, retval;
+ bool first_probe = false;
+
+ if (!idpf_adapter_list_init) {
+ rte_spinlock_init(&idpf_adapter_lock);
+ TAILQ_INIT(&idpf_adapter_list);
+ idpf_adapter_list_init = true;
+ }
+
+ adapter = idpf_find_adapter(pci_dev);
+ if (adapter == NULL) {
+ first_probe = true;
+ adapter = rte_zmalloc("idpf_adapter",
+ sizeof(struct idpf_adapter), 0);
+ if (adapter == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
+ return -ENOMEM;
+ }
+
+ retval = idpf_adapter_init(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init adapter.");
+ return retval;
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_INSERT_TAIL(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ }
+
+ retval = idpf_parse_devargs(pci_dev, adapter);
+ if (retval != 0) {
+ PMD_INIT_LOG(ERR, "Failed to parse private devargs");
+ goto err;
+ }
+
+ if (adapter->req_vport_nb == 0) {
+ /* If no vport devarg, create vport 0 by default. */
+ snprintf(name, sizeof(name), "idpf_%s_vport_0",
+ pci_dev->device.name);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create default vport 0");
+ adapter->cur_vports |= RTE_BIT32(0);
+ adapter->cur_vport_nb++;
+ } else {
+ for (i = 0; i < adapter->req_vport_nb; i++) {
+ snprintf(name, sizeof(name), "idpf_%s_vport_%d",
+ pci_dev->device.name,
+ adapter->req_vports[i]);
+ retval = rte_eth_dev_create(&pci_dev->device, name,
+ sizeof(struct idpf_vport),
+ NULL, NULL, idpf_dev_init,
+ adapter);
+ if (retval != 0)
+ PMD_DRV_LOG(ERR, "Failed to create vport %d",
+ adapter->req_vports[i]);
+ }
+ }
+
+ return 0;
+
+err:
+ if (first_probe) {
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+ }
+ return retval;
+}
+
+static int
+idpf_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct idpf_adapter *adapter = idpf_find_adapter(pci_dev);
+ uint16_t port_id;
+
+ /* Ethdev created can be found RTE_ETH_FOREACH_DEV_OF through rte_device */
+ RTE_ETH_FOREACH_DEV_OF(port_id, &pci_dev->device) {
+ rte_eth_dev_close(port_id);
+ }
+
+ rte_spinlock_lock(&idpf_adapter_lock);
+ TAILQ_REMOVE(&idpf_adapter_list, adapter, next);
+ rte_spinlock_unlock(&idpf_adapter_lock);
+ idpf_adapter_rel(adapter);
+ rte_free(adapter);
+
+ return 0;
+}
+
+static struct rte_pci_driver rte_idpf_pmd = {
+ .id_table = pci_id_idpf_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = idpf_pci_probe,
+ .remove = idpf_pci_remove,
+};
+
+/**
+ * Driver initialization routine.
+ * Invoked once at EAL init time.
+ * Register itself as the [Poll Mode] Driver of PCI devices.
+ */
+RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
+RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
+RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic | vfio-pci");
+
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
new file mode 100644
index 0000000000..84ae6641e2
--- /dev/null
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -0,0 +1,189 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_ETHDEV_H_
+#define _IDPF_ETHDEV_H_
+
+#include <stdint.h>
+#include <rte_malloc.h>
+#include <rte_spinlock.h>
+#include <rte_ethdev.h>
+#include <rte_kvargs.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+
+#include "idpf_logs.h"
+
+#include <base/idpf_prototype.h>
+#include <base/virtchnl2.h>
+
+#define IDPF_MAX_VPORT_NUM 8
+
+#define IDPF_DEFAULT_RXQ_NUM 16
+#define IDPF_DEFAULT_TXQ_NUM 16
+
+#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RX_BUFQ_PER_GRP 2
+
+#define IDPF_CTLQ_ID -1
+#define IDPF_CTLQ_LEN 64
+#define IDPF_DFLT_MBX_BUF_SIZE 4096
+
+#define IDPF_MIN_BUF_SIZE 1024
+#define IDPF_MAX_FRAME_SIZE 9728
+
+#define IDPF_NUM_MACADDR_MAX 64
+
+#define IDPF_VLAN_TAG_SIZE 4
+#define IDPF_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+
+#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
+
+/* Message type read in virtual channel from PF */
+enum idpf_vc_result {
+ IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
+ IDPF_MSG_NON, /* Read nothing from admin queue */
+ IDPF_MSG_SYS, /* Read system msg from admin queue */
+ IDPF_MSG_CMD, /* Read async command result */
+};
+
+struct idpf_chunks_info {
+ uint32_t tx_start_qid;
+ uint32_t rx_start_qid;
+ /* Valid only if split queue model */
+ uint32_t tx_compl_start_qid;
+ uint32_t rx_buf_start_qid;
+
+ uint64_t tx_qtail_start;
+ uint32_t tx_qtail_spacing;
+ uint64_t rx_qtail_start;
+ uint32_t rx_qtail_spacing;
+ uint64_t tx_compl_qtail_start;
+ uint32_t tx_compl_qtail_spacing;
+ uint64_t rx_buf_qtail_start;
+ uint32_t rx_buf_qtail_spacing;
+};
+
+struct idpf_vport {
+ struct idpf_adapter *adapter; /* Backreference to associated adapter */
+ uint16_t vport_id;
+ uint32_t txq_model;
+ uint32_t rxq_model;
+ uint16_t num_tx_q;
+ /* valid only if txq_model is split Q */
+ uint16_t num_tx_complq;
+ uint16_t num_rx_q;
+ /* valid only if rxq_model is split Q */
+ uint16_t num_rx_bufq;
+
+ uint16_t max_mtu;
+ uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+
+ uint16_t sw_idx; /* SW idx */
+
+ struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
+ uint16_t max_pkt_len; /* Maximum packet length */
+
+ /* Chunk info */
+ struct idpf_chunks_info chunks_info;
+
+ uint16_t devarg_id;
+};
+
+struct idpf_adapter {
+ TAILQ_ENTRY(idpf_adapter) next;
+ struct idpf_hw hw;
+ char name[IDPF_ADAPTER_NAME_LEN];
+
+ struct virtchnl2_version_info virtchnl_version;
+ struct virtchnl2_get_capabilities *caps;
+
+ volatile enum virtchnl_ops pend_cmd; /* pending command not finished */
+ uint32_t cmd_retval; /* return value of the cmd response from ipf */
+ uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf */
+
+ uint32_t txq_model; /* 0 - split queue model, non-0 - single queue model */
+ uint32_t rxq_model; /* 0 - split queue model, non-0 - single queue model */
+
+ /* Vport info */
+ uint8_t **vport_req_info;
+ uint8_t **vport_recv_info;
+ struct idpf_vport **vports;
+ uint16_t max_vport_nb;
+ uint16_t req_vports[IDPF_MAX_VPORT_NUM];
+ uint16_t req_vport_nb;
+ uint16_t cur_vports;
+ uint16_t cur_vport_nb;
+ uint16_t cur_vport_idx;
+
+ /* Max config queue number per VC message */
+ uint32_t max_rxq_per_msg;
+ uint32_t max_txq_per_msg;
+};
+
+TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
+
+#define IDPF_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+/* structure used for sending and checking response of virtchnl ops */
+struct idpf_cmd_info {
+ uint32_t ops;
+ uint8_t *in_args; /* buffer for sending */
+ uint32_t in_args_size; /* buffer size for sending */
+ uint8_t *out_buffer; /* buffer for response */
+ uint32_t out_size; /* buffer size for response */
+};
+
+/* notify current command done. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+notify_cmd(struct idpf_adapter *adapter, int msg_ret)
+{
+ adapter->cmd_retval = msg_ret;
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+}
+
+/* clear current command. Only call in case execute
+ * _atomic_set_cmd successfully.
+ */
+static inline void
+clear_cmd(struct idpf_adapter *adapter)
+{
+ /* Return value may be checked in anither thread, need to ensure the coherence. */
+ rte_wmb();
+ adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
+ adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS;
+}
+
+/* Check there is pending cmd in execution. If none, set new command. */
+static inline bool
+atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
+{
+ enum virtchnl_ops op_unk = VIRTCHNL_OP_UNKNOWN;
+ bool ret = __atomic_compare_exchange(&adapter->pend_cmd, &op_unk, &ops,
+ 0, __ATOMIC_ACQUIRE, __ATOMIC_ACQUIRE);
+
+ if (!ret)
+ PMD_DRV_LOG(ERR, "There is incomplete cmd %d", adapter->pend_cmd);
+
+ return !ret;
+}
+
+struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
+void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
+int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_vc_get_caps(struct idpf_adapter *adapter);
+int idpf_vc_create_vport(struct idpf_adapter *adapter);
+int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
+ uint16_t buf_len, uint8_t *buf);
+
+#endif /* _IDPF_ETHDEV_H_ */
diff --git a/drivers/net/idpf/idpf_logs.h b/drivers/net/idpf/idpf_logs.h
new file mode 100644
index 0000000000..d5f778fefe
--- /dev/null
+++ b/drivers/net/idpf/idpf_logs.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_LOGS_H_
+#define _IDPF_LOGS_H_
+
+#include <rte_log.h>
+
+extern int idpf_logtype_init;
+extern int idpf_logtype_driver;
+
+#define PMD_INIT_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_init, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG_RAW(level, ...) \
+ rte_log(RTE_LOG_ ## level, \
+ idpf_logtype_driver, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+#define PMD_DRV_LOG(level, fmt, args...) \
+ PMD_DRV_LOG_RAW(level, fmt "\n", ## args)
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_RX
+#define PMD_RX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_RX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#ifdef RTE_LIBRTE_IDPF_DEBUG_TX
+#define PMD_TX_LOG(level, ...) \
+ RTE_LOG(level, \
+ PMD, \
+ RTE_FMT("%s(): " \
+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ __func__, \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+#else
+#define PMD_TX_LOG(level, fmt, args...) do { } while (0)
+#endif
+
+#endif /* _IDPF_LOGS_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
new file mode 100644
index 0000000000..e89c2a2783
--- /dev/null
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#include <rte_debug.h>
+#include <rte_atomic.h>
+#include <rte_eal.h>
+#include <rte_ether.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_dev.h>
+
+#include "idpf_ethdev.h"
+
+static int
+idpf_vc_clean(struct idpf_adapter *adapter)
+{
+ struct idpf_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
+ uint16_t num_q_msg = IDPF_CTLQ_LEN;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+ uint32_t i;
+
+ for (i = 0; i < 10; i++) {
+ err = idpf_ctlq_clean_sq(adapter->hw.asq, &num_q_msg, q_msg);
+ msleep(20);
+ if (num_q_msg > 0)
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ /* Empty queue is not an error */
+ for (i = 0; i < num_q_msg; i++) {
+ dma_mem = q_msg[i]->ctx.indirect.payload;
+ if (dma_mem != NULL) {
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+ rte_free(dma_mem);
+ }
+ rte_free(q_msg[i]);
+ }
+
+ return 0;
+}
+
+static int
+idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
+ uint16_t msg_size, uint8_t *msg)
+{
+ struct idpf_ctlq_msg *ctlq_msg;
+ struct idpf_dma_mem *dma_mem;
+ int err;
+
+ err = idpf_vc_clean(adapter);
+ if (err != 0)
+ goto err;
+
+ ctlq_msg = rte_zmalloc(NULL, sizeof(struct idpf_ctlq_msg), 0);
+ if (ctlq_msg == NULL) {
+ err = -ENOMEM;
+ goto err;
+ }
+
+ dma_mem = rte_zmalloc(NULL, sizeof(struct idpf_dma_mem), 0);
+ if (dma_mem == NULL) {
+ err = -ENOMEM;
+ goto dma_mem_error;
+ }
+
+ dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
+ idpf_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem->size);
+ if (dma_mem->va == NULL) {
+ err = -ENOMEM;
+ goto dma_alloc_error;
+ }
+
+ memcpy(dma_mem->va, msg, msg_size);
+
+ ctlq_msg->opcode = idpf_mbq_opc_send_msg_to_pf;
+ ctlq_msg->func_id = 0;
+ ctlq_msg->data_len = msg_size;
+ ctlq_msg->cookie.mbx.chnl_opcode = op;
+ ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
+ ctlq_msg->ctx.indirect.payload = dma_mem;
+
+ err = idpf_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
+ if (err != 0)
+ goto send_error;
+
+ return 0;
+
+send_error:
+ idpf_free_dma_mem(&adapter->hw, dma_mem);
+dma_alloc_error:
+ rte_free(dma_mem);
+dma_mem_error:
+ rte_free(ctlq_msg);
+err:
+ return err;
+}
+
+static enum idpf_vc_result
+idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
+ uint8_t *buf)
+{
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_ctlq_msg ctlq_msg;
+ struct idpf_dma_mem *dma_mem = NULL;
+ enum idpf_vc_result result = IDPF_MSG_NON;
+ enum virtchnl_ops opcode;
+ uint16_t pending = 1;
+ int ret;
+
+ ret = idpf_ctlq_recv(hw->arq, &pending, &ctlq_msg);
+ if (ret != 0) {
+ PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
+ if (ret != -ENOMSG)
+ result = IDPF_MSG_ERR;
+ return result;
+ }
+
+ rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+
+ opcode = (enum virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
+ adapter->cmd_retval =
+ (enum virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
+
+ PMD_DRV_LOG(DEBUG, "CQ from CP carries opcode %u, retval %d",
+ opcode, adapter->cmd_retval);
+
+ if (opcode == VIRTCHNL2_OP_EVENT) {
+ struct virtchnl2_event *ve =
+ (struct virtchnl2_event *)ctlq_msg.ctx.indirect.payload->va;
+
+ result = IDPF_MSG_SYS;
+ switch (ve->event) {
+ case VIRTCHNL2_EVENT_LINK_CHANGE:
+ /* TBD */
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "%s: Unknown event %d from CP",
+ __func__, ve->event);
+ break;
+ }
+ } else {
+ /* async reply msg on command issued by pf previously */
+ result = IDPF_MSG_CMD;
+ if (opcode != adapter->pend_cmd) {
+ PMD_DRV_LOG(WARNING, "command mismatch, expect %u, get %u",
+ adapter->pend_cmd, opcode);
+ result = IDPF_MSG_ERR;
+ }
+ }
+
+ if (ctlq_msg.data_len != 0)
+ dma_mem = ctlq_msg.ctx.indirect.payload;
+ else
+ pending = 0;
+
+ ret = idpf_ctlq_post_rx_buffs(hw, hw->arq, &pending, &dma_mem);
+ if (ret != 0 && dma_mem != NULL)
+ idpf_free_dma_mem(hw, dma_mem);
+
+ return result;
+}
+
+#define MAX_TRY_TIMES 200
+#define ASQ_DELAY_MS 10
+
+int
+idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops, uint16_t buf_len,
+ uint8_t *buf)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = idpf_read_msg_from_cp(adapter, buf_len, buf);
+ if (ret == IDPF_MSG_CMD)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ } while (i++ < MAX_TRY_TIMES);
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, ops);
+ }
+
+ return err;
+}
+
+static int
+idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
+{
+ int err = 0;
+ int i = 0;
+ int ret;
+
+ if (atomic_set_cmd(adapter, args->ops))
+ return -EINVAL;
+
+ ret = idpf_send_vc_msg(adapter, args->ops, args->in_args_size, args->in_args);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
+ clear_cmd(adapter);
+ return ret;
+ }
+
+ switch (args->ops) {
+ case VIRTCHNL_OP_VERSION:
+ case VIRTCHNL2_OP_GET_CAPS:
+ case VIRTCHNL2_OP_CREATE_VPORT:
+ case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_ENABLE_VPORT:
+ case VIRTCHNL2_OP_DISABLE_VPORT:
+ /* for init virtchnl ops, need to poll the response */
+ err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
+ clear_cmd(adapter);
+ break;
+ default:
+ /* For other virtchnl ops in running time,
+ * wait for the cmd done flag.
+ */
+ do {
+ if (adapter->pend_cmd == VIRTCHNL_OP_UNKNOWN)
+ break;
+ rte_delay_ms(ASQ_DELAY_MS);
+ /* If don't read msg or read sys event, continue */
+ } while (i++ < MAX_TRY_TIMES);
+ /* If there's no response is received, clear command */
+ if (i >= MAX_TRY_TIMES ||
+ adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
+ err = -EBUSY;
+ PMD_DRV_LOG(ERR, "No response or return failure (%d) for cmd %d",
+ adapter->cmd_retval, args->ops);
+ clear_cmd(adapter);
+ }
+ break;
+ }
+
+ return err;
+}
+
+int
+idpf_vc_check_api_version(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_version_info version, *pver;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&version, 0, sizeof(struct virtchnl_version_info));
+ version.major = VIRTCHNL2_VERSION_MAJOR_2;
+ version.minor = VIRTCHNL2_VERSION_MINOR_0;
+
+ args.ops = VIRTCHNL_OP_VERSION;
+ args.in_args = (uint8_t *)&version;
+ args.in_args_size = sizeof(version);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL_OP_VERSION");
+ return err;
+ }
+
+ pver = (struct virtchnl2_version_info *)args.out_buffer;
+ adapter->virtchnl_version = *pver;
+
+ if (adapter->virtchnl_version.major != VIRTCHNL2_VERSION_MAJOR_2 ||
+ adapter->virtchnl_version.minor != VIRTCHNL2_VERSION_MINOR_0) {
+ PMD_INIT_LOG(ERR, "VIRTCHNL API version mismatch:(%u.%u)-(%u.%u)",
+ adapter->virtchnl_version.major,
+ adapter->virtchnl_version.minor,
+ VIRTCHNL2_VERSION_MAJOR_2,
+ VIRTCHNL2_VERSION_MINOR_0);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int
+idpf_vc_get_caps(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_capabilities caps_msg;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+
+ args.ops = VIRTCHNL2_OP_GET_CAPS;
+ args.in_args = (uint8_t *)&caps_msg;
+ args.in_args_size = sizeof(caps_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_GET_CAPS");
+ return err;
+ }
+
+ rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
+
+ return 0;
+}
+
+int
+idpf_vc_create_vport(struct idpf_adapter *adapter)
+{
+ uint16_t idx = adapter->cur_vport_idx;
+ struct virtchnl2_create_vport *vport_req_info =
+ (struct virtchnl2_create_vport *)adapter->vport_req_info[idx];
+ struct virtchnl2_create_vport vport_msg;
+ struct idpf_cmd_info args;
+ int err = -1;
+
+ memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
+ vport_msg.vport_type = vport_req_info->vport_type;
+ vport_msg.txq_model = vport_req_info->txq_model;
+ vport_msg.rxq_model = vport_req_info->rxq_model;
+ vport_msg.num_tx_q = vport_req_info->num_tx_q;
+ vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
+ vport_msg.num_rx_q = vport_req_info->num_rx_q;
+ vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CREATE_VPORT;
+ args.in_args = (uint8_t *)&vport_msg;
+ args.in_args_size = sizeof(vport_msg);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Failed to execute command of VIRTCHNL2_OP_CREATE_VPORT");
+ return err;
+ }
+
+ if (adapter->vport_recv_info[idx] == NULL) {
+ adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
+ IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (adapter->vport_recv_info[idx] == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to alloc vport_recv_info.");
+ return -ENOMEM;
+ }
+ }
+ rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
+ IDPF_DFLT_MBX_BUF_SIZE);
+ return 0;
+}
+
+int
+idpf_vc_destroy_vport(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_DESTROY_VPORT");
+
+ return err;
+}
+
+int
+idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_vport vc_vport;
+ struct idpf_cmd_info args;
+ int err;
+
+ vc_vport.vport_id = vport->vport_id;
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
+ VIRTCHNL2_OP_DISABLE_VPORT;
+ args.in_args = (uint8_t *)&vc_vport;
+ args.in_args_size = sizeof(vc_vport);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_VPORT",
+ enable ? "ENABLE" : "DISABLE");
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
new file mode 100644
index 0000000000..ecf73355c3
--- /dev/null
+++ b/drivers/net/idpf/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+if is_windows
+ build = false
+ reason = 'not supported on Windows'
+ subdir_done()
+endif
+
+deps += ['common_idpf']
+
+sources = files(
+ 'idpf_ethdev.c',
+ 'idpf_vchnl.c',
+)
diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
new file mode 100644
index 0000000000..78c3585d7c
--- /dev/null
+++ b/drivers/net/idpf/version.map
@@ -0,0 +1,3 @@
+DPDK_23 {
+ local: *;
+};
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 589399400a..6470bf3636 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -29,6 +29,7 @@ drivers = [
'i40e',
'iavf',
'ice',
+ 'idpf',
'igc',
'ionic',
'ipn3ke',
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 03/18] net/idpf: add Tx queue setup
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 8:33 ` [PATCH v18 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 8:33 ` [PATCH v18 02/18] net/idpf: add support for device initialization beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 04/18] net/idpf: add Rx " beilei.xing
` (15 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for tx_queue_setup ops.
In the single queue model, the same descriptor queue is used by SW to
post buffer descriptors to HW and by HW to post completed descriptors
to SW.
In the split queue model, "RX buffer queues" are used to pass
descriptor buffers from SW to HW while Rx queues are used only to
pass the descriptor completions, that is, descriptors that point
to completed buffers, from HW to SW. This is contrary to the single
queue model in which Rx queues are used for both purposes.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 13 ++
drivers/net/idpf/idpf_rxtx.c | 364 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 70 +++++++
drivers/net/idpf/meson.build | 1 +
4 files changed, 448 insertions(+)
create mode 100644 drivers/net/idpf/idpf_rxtx.c
create mode 100644 drivers/net/idpf/idpf_rxtx.h
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 035f563275..54f20d30ca 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -11,6 +11,7 @@
#include <errno.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
#define IDPF_TX_SINGLE_Q "tx_single"
#define IDPF_RX_SINGLE_Q "rx_single"
@@ -42,6 +43,17 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -631,6 +643,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
new file mode 100644
index 0000000000..4afa0a2560
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -0,0 +1,364 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+static int
+check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
+ uint16_t tx_free_thresh)
+{
+ /* TX descriptors will have their RS bit set after tx_rs_thresh
+ * descriptors have been used. The TX descriptor ring will be cleaned
+ * after tx_free_thresh descriptors are used or if the number of
+ * descriptors required to transmit a packet is greater than the
+ * number of free TX descriptors.
+ *
+ * The following constraints must be satisfied:
+ * - tx_rs_thresh must be less than the size of the ring minus 2.
+ * - tx_free_thresh must be less than the size of the ring minus 3.
+ * - tx_rs_thresh must be less than or equal to tx_free_thresh.
+ * - tx_rs_thresh must be a divisor of the ring size.
+ *
+ * One descriptor in the TX ring is used as a sentinel to avoid a H/W
+ * race condition, hence the maximum threshold constraints. When set
+ * to zero use default values.
+ */
+ if (tx_rs_thresh >= (nb_desc - 2)) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 2",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_free_thresh >= (nb_desc - 3)) {
+ PMD_INIT_LOG(ERR, "tx_free_thresh (%u) must be less than the "
+ "number of TX descriptors (%u) minus 3.",
+ tx_free_thresh, nb_desc);
+ return -EINVAL;
+ }
+ if (tx_rs_thresh > tx_free_thresh) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be less than or "
+ "equal to tx_free_thresh (%u).",
+ tx_rs_thresh, tx_free_thresh);
+ return -EINVAL;
+ }
+ if ((nb_desc % tx_rs_thresh) != 0) {
+ PMD_INIT_LOG(ERR, "tx_rs_thresh (%u) must be a divisor of the "
+ "number of TX descriptors (%u).",
+ tx_rs_thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void
+reset_split_tx_descq(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->desc_ring)[i] = 0;
+
+ txe = txq->sw_ring;
+ prev = (uint16_t)(txq->sw_nb_desc - 1);
+ for (i = 0; i < txq->sw_nb_desc; i++) {
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ /* Use this as next to clean for split desc queue */
+ txq->last_desc_cleaned = 0;
+ txq->sw_tail = 0;
+ txq->nb_free = txq->nb_tx_desc - 1;
+}
+
+static void
+reset_split_tx_complq(struct idpf_tx_queue *cq)
+{
+ uint32_t i, size;
+
+ if (cq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to complq is NULL");
+ return;
+ }
+
+ size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)cq->compl_ring)[i] = 0;
+
+ cq->tx_tail = 0;
+ cq->expected_gen_id = 1;
+}
+
+static void
+reset_single_tx_queue(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_entry *txe;
+ uint32_t i, size;
+ uint16_t prev;
+
+ if (txq == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to txq is NULL");
+ return;
+ }
+
+ txe = txq->sw_ring;
+ size = sizeof(struct idpf_flex_tx_desc) * txq->nb_tx_desc;
+ for (i = 0; i < size; i++)
+ ((volatile char *)txq->tx_ring)[i] = 0;
+
+ prev = (uint16_t)(txq->nb_tx_desc - 1);
+ for (i = 0; i < txq->nb_tx_desc; i++) {
+ txq->tx_ring[i].qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE);
+ txe[i].mbuf = NULL;
+ txe[i].last_id = i;
+ txe[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->tx_tail = 0;
+ txq->nb_used = 0;
+
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ txq->nb_free = txq->nb_tx_desc - 1;
+
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+static int
+idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ struct idpf_tx_queue *txq, *cq;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint64_t offloads;
+ int ret;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh != 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh != 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf split txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_nb_desc = 2 * nb_desc;
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf split tx sw ring",
+ sizeof(struct idpf_tx_entry) *
+ txq->sw_nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ ret = -ENOMEM;
+ goto err_txq_sw_ring;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_sched_desc) * txq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "split_tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ ret = -ENOMEM;
+ goto err_txq_mz;
+ }
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->desc_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_split_tx_descq(txq);
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ /* Allocate the TX completion queue data structure. */
+ txq->complq = rte_zmalloc_socket("idpf splitq cq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ cq = txq->complq;
+ if (cq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ ret = -ENOMEM;
+ goto err_cq;
+ }
+ cq->nb_tx_desc = 2 * nb_desc;
+ cq->queue_id = vport->chunks_info.tx_compl_start_qid + queue_idx;
+ cq->port_id = dev->data->port_id;
+ cq->txqs = dev->data->tx_queues;
+ cq->tx_start_qid = vport->chunks_info.tx_start_qid;
+
+ ring_size = sizeof(struct idpf_splitq_tx_compl_desc) * cq->nb_tx_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_split_compl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX completion queue");
+ ret = -ENOMEM;
+ goto err_cq_mz;
+ }
+ cq->tx_ring_phys_addr = mz->iova;
+ cq->compl_ring = mz->addr;
+ cq->mz = mz;
+ reset_split_tx_complq(cq);
+
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+
+ return 0;
+
+err_cq_mz:
+ rte_free(cq);
+err_cq:
+ rte_memzone_free(txq->mz);
+err_txq_mz:
+ rte_free(txq->sw_ring);
+err_txq_sw_ring:
+ rte_free(txq);
+
+ return ret;
+}
+
+static int
+idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t tx_rs_thresh, tx_free_thresh;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_tx_queue *txq;
+ uint32_t ring_size;
+ uint64_t offloads;
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ tx_rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh > 0) ?
+ tx_conf->tx_rs_thresh : IDPF_DEFAULT_TX_RS_THRESH);
+ tx_free_thresh = (uint16_t)((tx_conf->tx_free_thresh > 0) ?
+ tx_conf->tx_free_thresh : IDPF_DEFAULT_TX_FREE_THRESH);
+ if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Allocate the TX queue data structure. */
+ txq = rte_zmalloc_socket("idpf txq",
+ sizeof(struct idpf_tx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for tx queue structure");
+ return -ENOMEM;
+ }
+
+ /* TODO: vlan offload */
+
+ txq->nb_tx_desc = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+
+ /* Allocate software ring */
+ txq->sw_ring =
+ rte_zmalloc_socket("idpf tx sw ring",
+ sizeof(struct idpf_tx_entry) * nb_desc,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (txq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW TX ring");
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ /* Allocate TX hardware ring descriptors. */
+ ring_size = sizeof(struct idpf_flex_tx_desc) * nb_desc;
+ ring_size = RTE_ALIGN(ring_size, IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for TX");
+ rte_free(txq->sw_ring);
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring_phys_addr = mz->iova;
+ txq->tx_ring = mz->addr;
+
+ txq->mz = mz;
+ reset_single_tx_queue(txq);
+ txq->q_set = true;
+ dev->data->tx_queues[queue_idx] = txq;
+ txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
+ queue_idx * vport->chunks_info.tx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_tx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+ else
+ return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, tx_conf);
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
new file mode 100644
index 0000000000..c7ba15b058
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_H_
+#define _IDPF_RXTX_H_
+
+#include "idpf_ethdev.h"
+
+/* In QLEN must be whole number of 32 descriptors. */
+#define IDPF_ALIGN_RING_DESC 32
+#define IDPF_MIN_RING_DESC 32
+#define IDPF_MAX_RING_DESC 4096
+#define IDPF_DMA_MEM_ALIGN 4096
+/* Base address of the HW descriptor ring should be 128B aligned. */
+#define IDPF_RING_BASE_ALIGN 128
+
+#define IDPF_DEFAULT_TX_RS_THRESH 32
+#define IDPF_DEFAULT_TX_FREE_THRESH 32
+
+struct idpf_tx_entry {
+ struct rte_mbuf *mbuf;
+ uint16_t next_id;
+ uint16_t last_id;
+};
+
+/* Structure associated with each TX queue. */
+struct idpf_tx_queue {
+ const struct rte_memzone *mz; /* memzone for Tx ring */
+ volatile struct idpf_flex_tx_desc *tx_ring; /* Tx ring virtual address */
+ volatile union {
+ struct idpf_flex_tx_sched_desc *desc_ring;
+ struct idpf_splitq_tx_compl_desc *compl_ring;
+ };
+ uint64_t tx_ring_phys_addr; /* Tx ring DMA address */
+ struct idpf_tx_entry *sw_ring; /* address array of SW ring */
+
+ uint16_t nb_tx_desc; /* ring length */
+ uint16_t tx_tail; /* current value of tail */
+ volatile uint8_t *qtx_tail; /* register address of tail */
+ /* number of used desc since RS bit set */
+ uint16_t nb_used;
+ uint16_t nb_free;
+ uint16_t last_desc_cleaned; /* last desc have been cleaned*/
+ uint16_t free_thresh;
+ uint16_t rs_thresh;
+
+ uint16_t port_id;
+ uint16_t queue_id;
+ uint64_t offloads;
+ uint16_t next_dd; /* next to set RS, for VPMD */
+ uint16_t next_rs; /* next to check DD, for VPMD */
+
+ bool q_set; /* if tx queue has been configured */
+ bool q_started; /* if tx queue has been started */
+
+ /* only valid for split queue mode */
+ uint16_t sw_nb_desc;
+ uint16_t sw_tail;
+ void **txqs;
+ uint32_t tx_start_qid;
+ uint8_t expected_gen_id;
+ struct idpf_tx_queue *complq;
+};
+
+int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index ecf73355c3..b632b76656 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -11,5 +11,6 @@ deps += ['common_idpf']
sources = files(
'idpf_ethdev.c',
+ 'idpf_rxtx.c',
'idpf_vchnl.c',
)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 04/18] net/idpf: add Rx queue setup
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (2 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 03/18] net/idpf: add Tx queue setup beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 05/18] net/idpf: add support for device start and stop beilei.xing
` (14 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for rx_queue_setup ops.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 11 +
drivers/net/idpf/idpf_rxtx.c | 400 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 46 ++++
3 files changed, 457 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 54f20d30ca..fb5cd1b111 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -48,12 +48,22 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
};
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_free_thresh = IDPF_DEFAULT_RX_FREE_THRESH,
+ };
+
dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
.nb_max = IDPF_MAX_RING_DESC,
.nb_min = IDPF_MIN_RING_DESC,
.nb_align = IDPF_ALIGN_RING_DESC,
};
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = IDPF_MAX_RING_DESC,
+ .nb_min = IDPF_MIN_RING_DESC,
+ .nb_align = IDPF_ALIGN_RING_DESC,
+ };
+
return 0;
}
@@ -643,6 +653,7 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_configure = idpf_dev_configure,
.dev_close = idpf_dev_close,
+ .rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
};
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 4afa0a2560..25dd5d85d5 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -8,6 +8,21 @@
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+static int
+check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
+{
+ /* The following constraints must be satisfied:
+ * thresh < rxq->nb_rx_desc
+ */
+ if (thresh >= nb_desc) {
+ PMD_INIT_LOG(ERR, "rx_free_thresh (%u) must be less than %u",
+ thresh, nb_desc);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int
check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
uint16_t tx_free_thresh)
@@ -56,6 +71,87 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+reset_split_rx_descq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ rxq->rx_tail = 0;
+ rxq->expected_gen_id = 1;
+}
+
+static void
+reset_split_rx_bufq(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_splitq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ /* The next descriptor id which can be received. */
+ rxq->rx_next_avail = 0;
+
+ /* The next descriptor id which can be refilled. */
+ rxq->rx_tail = 0;
+ /* The number of descriptors which can be refilled. */
+ rxq->nb_rx_hold = rxq->nb_rx_desc - 1;
+
+ rxq->bufq1 = NULL;
+ rxq->bufq2 = NULL;
+}
+
+static void
+reset_single_rx_queue(struct idpf_rx_queue *rxq)
+{
+ uint16_t len;
+ uint32_t i;
+
+ if (rxq == NULL)
+ return;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+
+ for (i = 0; i < len * sizeof(struct virtchnl2_singleq_rx_buf_desc);
+ i++)
+ ((volatile char *)rxq->rx_ring)[i] = 0;
+
+ memset(&rxq->fake_mbuf, 0x0, sizeof(rxq->fake_mbuf));
+
+ for (i = 0; i < IDPF_RX_MAX_BURST; i++)
+ rxq->sw_ring[rxq->nb_rx_desc + i] = &rxq->fake_mbuf;
+
+ rxq->rx_tail = 0;
+ rxq->nb_rx_hold = 0;
+
+ if (rxq->pkt_first_seg != NULL)
+ rte_pktmbuf_free(rxq->pkt_first_seg);
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+}
+
static void
reset_split_tx_descq(struct idpf_tx_queue *txq)
{
@@ -145,6 +241,310 @@ reset_single_tx_queue(struct idpf_tx_queue *txq)
txq->next_rs = txq->rs_thresh - 1;
}
+static int
+idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
+ uint16_t queue_idx, uint16_t rx_free_thresh,
+ uint16_t nb_desc, unsigned int socket_id,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ uint32_t ring_size;
+ uint16_t len;
+
+ bufq->mp = mp;
+ bufq->nb_rx_desc = nb_desc;
+ bufq->rx_free_thresh = rx_free_thresh;
+ bufq->queue_id = vport->chunks_info.rx_buf_start_qid + queue_idx;
+ bufq->port_id = dev->data->port_id;
+ bufq->rx_hdr_len = 0;
+ bufq->adapter = adapter;
+
+ len = rte_pktmbuf_data_room_size(bufq->mp) - RTE_PKTMBUF_HEADROOM;
+ bufq->rx_buf_len = len;
+
+ /* Allocate the software ring. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ bufq->sw_ring =
+ rte_zmalloc_socket("idpf rx bufq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_splitq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_buf_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(bufq->sw_ring);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ bufq->rx_ring_phys_addr = mz->iova;
+ bufq->rx_ring = mz->addr;
+
+ bufq->mz = mz;
+ reset_split_rx_bufq(bufq);
+ bufq->q_set = true;
+ bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
+ queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+
+ /* TODO: allow bulk or vec */
+
+ return 0;
+}
+
+static int
+idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue *bufq1, *bufq2;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t qid;
+ uint16_t len;
+ int ret;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = rxq->nb_rx_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_rx_flex_desc_adv_nic_3),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_cpmpl_ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX");
+ ret = -ENOMEM;
+ goto free_rxq;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_split_rx_descq(rxq);
+
+ /* TODO: allow bulk or vec */
+
+ /* setup Rx buffer queue */
+ bufq1 = rte_zmalloc_socket("idpf bufq1",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq1 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 1.");
+ ret = -ENOMEM;
+ goto free_mz;
+ }
+ qid = 2 * queue_idx;
+ ret = idpf_rx_split_bufq_setup(dev, bufq1, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 1");
+ ret = -EINVAL;
+ goto free_bufq1;
+ }
+ rxq->bufq1 = bufq1;
+
+ bufq2 = rte_zmalloc_socket("idpf bufq2",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (bufq2 == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx buffer queue 2.");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -ENOMEM;
+ goto free_bufq1;
+ }
+ qid = 2 * queue_idx + 1;
+ ret = idpf_rx_split_bufq_setup(dev, bufq2, qid, rx_free_thresh,
+ nb_desc, socket_id, mp);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to setup buffer queue 2");
+ rte_free(bufq1->sw_ring);
+ rte_memzone_free(bufq1->mz);
+ ret = -EINVAL;
+ goto free_bufq2;
+ }
+ rxq->bufq2 = bufq2;
+
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ return 0;
+
+free_bufq2:
+ rte_free(bufq2);
+free_bufq1:
+ rte_free(bufq1);
+free_mz:
+ rte_memzone_free(mz);
+free_rxq:
+ rte_free(rxq);
+
+ return ret;
+}
+
+static int
+idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_hw *hw = &adapter->hw;
+ const struct rte_memzone *mz;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_free_thresh;
+ uint32_t ring_size;
+ uint64_t offloads;
+ uint16_t len;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ /* Check free threshold */
+ rx_free_thresh = (rx_conf->rx_free_thresh == 0) ?
+ IDPF_DEFAULT_RX_FREE_THRESH :
+ rx_conf->rx_free_thresh;
+ if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
+ return -EINVAL;
+
+ /* Setup Rx description queue */
+ rxq = rte_zmalloc_socket("idpf rxq",
+ sizeof(struct idpf_rx_queue),
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for rx queue data structure");
+ return -ENOMEM;
+ }
+
+ rxq->mp = mp;
+ rxq->nb_rx_desc = nb_desc;
+ rxq->rx_free_thresh = rx_free_thresh;
+ rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
+ rxq->port_id = dev->data->port_id;
+ rxq->rx_hdr_len = 0;
+ rxq->adapter = adapter;
+ rxq->offloads = offloads;
+
+ len = rte_pktmbuf_data_room_size(rxq->mp) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = len;
+
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ rxq->sw_ring =
+ rte_zmalloc_socket("idpf rxq sw ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (rxq->sw_ring == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate memory for SW ring");
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Allocate a liitle more to support bulk allocate. */
+ len = nb_desc + IDPF_RX_MAX_BURST;
+ ring_size = RTE_ALIGN(len *
+ sizeof(struct virtchnl2_singleq_rx_buf_desc),
+ IDPF_DMA_MEM_ALIGN);
+ mz = rte_eth_dma_zone_reserve(dev, "rx ring", queue_idx,
+ ring_size, IDPF_RING_BASE_ALIGN,
+ socket_id);
+ if (mz == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to reserve DMA memory for RX buffer queue.");
+ rte_free(rxq->sw_ring);
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ /* Zero all the descriptors in the ring. */
+ memset(mz->addr, 0, ring_size);
+ rxq->rx_ring_phys_addr = mz->iova;
+ rxq->rx_ring = mz->addr;
+
+ rxq->mz = mz;
+ reset_single_rx_queue(rxq);
+ rxq->q_set = true;
+ dev->data->rx_queues[queue_idx] = rxq;
+ rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
+ queue_idx * vport->chunks_info.rx_qtail_spacing);
+
+ return 0;
+}
+
+int
+idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ return idpf_rx_single_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+ else
+ return idpf_rx_split_queue_setup(dev, queue_idx, nb_desc,
+ socket_id, rx_conf, mp);
+}
+
static int
idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index c7ba15b058..3f3932c3eb 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -15,9 +15,51 @@
/* Base address of the HW descriptor ring should be 128B aligned. */
#define IDPF_RING_BASE_ALIGN 128
+#define IDPF_RX_MAX_BURST 32
+#define IDPF_DEFAULT_RX_FREE_THRESH 32
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+struct idpf_rx_queue {
+ struct idpf_adapter *adapter; /* the adapter this queue belongs to */
+ struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
+ const struct rte_memzone *mz; /* memzone for Rx ring */
+ volatile void *rx_ring;
+ struct rte_mbuf **sw_ring; /* address of SW ring */
+ uint64_t rx_ring_phys_addr; /* Rx ring DMA address */
+
+ uint16_t nb_rx_desc; /* ring length */
+ uint16_t rx_tail; /* current value of tail */
+ volatile uint8_t *qrx_tail; /* register address of tail */
+ uint16_t rx_free_thresh; /* max free RX desc to hold */
+ uint16_t nb_rx_hold; /* number of held free RX desc */
+ struct rte_mbuf *pkt_first_seg; /* first segment of current packet */
+ struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
+ struct rte_mbuf fake_mbuf; /* dummy mbuf */
+
+ uint16_t rx_nb_avail;
+ uint16_t rx_next_avail;
+
+ uint16_t port_id; /* device port ID */
+ uint16_t queue_id; /* Rx queue index */
+ uint16_t rx_buf_len; /* The packet buffer size */
+ uint16_t rx_hdr_len; /* The header buffer size */
+ uint16_t max_pkt_len; /* Maximum packet length */
+ uint8_t rxdid;
+
+ bool q_set; /* if rx queue has been configured */
+ bool q_started; /* if rx queue has been started */
+
+ /* only valid for split queue mode */
+ uint8_t expected_gen_id;
+ struct idpf_rx_queue *bufq1;
+ struct idpf_rx_queue *bufq2;
+
+ uint64_t offloads;
+ uint32_t hw_register_set;
+};
+
struct idpf_tx_entry {
struct rte_mbuf *mbuf;
uint16_t next_id;
@@ -63,6 +105,10 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t nb_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 05/18] net/idpf: add support for device start and stop
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (3 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 04/18] net/idpf: add Rx " beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 06/18] net/idpf: add support for queue start beilei.xing
` (13 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops dev_start, dev_stop and link_update.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 55 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.c | 20 +++++++++++++
2 files changed, 75 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index fb5cd1b111..621bf9aad5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -29,6 +29,22 @@ static const char * const idpf_valid_args[] = {
NULL
};
+static int
+idpf_dev_link_update(struct rte_eth_dev *dev,
+ __rte_unused int wait_to_complete)
+{
+ struct rte_eth_link new_link;
+
+ memset(&new_link, 0, sizeof(new_link));
+
+ new_link.link_speed = RTE_ETH_SPEED_NUM_NONE;
+ new_link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ new_link.link_autoneg = !(dev->data->dev_conf.link_speeds &
+ RTE_ETH_LINK_SPEED_FIXED);
+
+ return rte_eth_linkstatus_set(dev, &new_link);
+}
+
static int
idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
{
@@ -267,6 +283,42 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_dev_start(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ int ret;
+
+ if (dev->data->mtu > vport->max_mtu) {
+ PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
+ return -EINVAL;
+ }
+
+ vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+
+ /* TODO: start queues */
+
+ ret = idpf_vc_ena_dis_vport(vport, true);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to enable vport");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+idpf_dev_stop(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ idpf_vc_ena_dis_vport(vport, false);
+
+ /* TODO: stop queues */
+
+ return 0;
+}
+
static int
idpf_dev_close(struct rte_eth_dev *dev)
{
@@ -656,6 +708,9 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.rx_queue_setup = idpf_rx_queue_setup,
.tx_queue_setup = idpf_tx_queue_setup,
.dev_infos_get = idpf_dev_info_get,
+ .dev_start = idpf_dev_start,
+ .dev_stop = idpf_dev_stop,
+ .link_update = idpf_dev_link_update,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 25dd5d85d5..3528d2f2c7 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,6 +334,11 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -465,6 +470,11 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ if (rx_conf->rx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -569,6 +579,11 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -691,6 +706,11 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ if (tx_conf->tx_deferred_start) {
+ PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
+ return -EINVAL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 06/18] net/idpf: add support for queue start
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (4 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 05/18] net/idpf: add support for device start and stop beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 07/18] net/idpf: add support for queue stop beilei.xing
` (12 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_start
- tx_queue_start
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 42 +++-
drivers/net/idpf/idpf_ethdev.h | 9 +
drivers/net/idpf/idpf_rxtx.c | 237 +++++++++++++++--
drivers/net/idpf/idpf_rxtx.h | 6 +
drivers/net/idpf/idpf_vchnl.c | 447 +++++++++++++++++++++++++++++++++
5 files changed, 720 insertions(+), 21 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 621bf9aad5..0400ed611f 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -283,6 +283,39 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_start_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int err = 0;
+ int i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL || txq->tx_deferred_start)
+ continue;
+ err = idpf_tx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Tx queue %u", i);
+ return err;
+ }
+ }
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL || rxq->rx_deferred_start)
+ continue;
+ err = idpf_rx_queue_start(dev, i);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to start Rx queue %u", i);
+ return err;
+ }
+ }
+
+ return err;
+}
+
static int
idpf_dev_start(struct rte_eth_dev *dev)
{
@@ -296,11 +329,16 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
- /* TODO: start queues */
+ ret = idpf_start_queues(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to start queues");
+ return ret;
+ }
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
+ /* TODO: stop queues */
return ret;
}
@@ -711,6 +749,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.dev_start = idpf_dev_start,
.dev_stop = idpf_dev_stop,
.link_update = idpf_dev_link_update,
+ .rx_queue_start = idpf_rx_queue_start,
+ .tx_queue_start = idpf_tx_queue_start,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 84ae6641e2..96c22009e9 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -24,7 +24,9 @@
#define IDPF_DEFAULT_TXQ_NUM 16
#define IDPF_INVALID_VPORT_IDX 0xffff
+#define IDPF_TXQ_PER_GRP 1
#define IDPF_TX_COMPLQ_PER_GRP 1
+#define IDPF_RXQ_PER_GRP 1
#define IDPF_RX_BUFQ_PER_GRP 2
#define IDPF_CTLQ_ID -1
@@ -182,6 +184,13 @@ int idpf_vc_check_api_version(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_config_rxqs(struct idpf_vport *vport);
+int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
+int idpf_vc_config_txqs(struct idpf_vport *vport);
+int idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id);
+int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on);
+int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 3528d2f2c7..6d954afd9d 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -334,11 +334,6 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -354,6 +349,7 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -470,11 +466,6 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
- if (rx_conf->rx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -490,6 +481,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
rxq->rx_free_thresh = rx_free_thresh;
rxq->queue_id = vport->chunks_info.rx_start_qid + queue_idx;
rxq->port_id = dev->data->port_id;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
rxq->rx_hdr_len = 0;
rxq->adapter = adapter;
rxq->offloads = offloads;
@@ -579,11 +571,6 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -600,6 +587,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_nb_desc = 2 * nb_desc;
@@ -706,11 +694,6 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
- if (tx_conf->tx_deferred_start) {
- PMD_INIT_LOG(ERR, "Queue start is not supported currently.");
- return -EINVAL;
- }
-
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -729,6 +712,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
txq->queue_id = vport->chunks_info.tx_start_qid + queue_idx;
txq->port_id = dev->data->port_id;
txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
/* Allocate software ring */
txq->sw_ring =
@@ -782,3 +766,216 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+static int
+idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_singleq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_singleq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd1 = 0;
+ rxd->rsvd2 = 0;
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ return 0;
+}
+
+static int
+idpf_alloc_split_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rxd;
+ struct rte_mbuf *mbuf = NULL;
+ uint64_t dma_addr;
+ uint16_t i;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(mbuf == NULL)) {
+ PMD_DRV_LOG(ERR, "Failed to allocate mbuf for RX");
+ return -ENOMEM;
+ }
+
+ rte_mbuf_refcnt_set(mbuf, 1);
+ mbuf->next = NULL;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+
+ rxd = &((volatile struct virtchnl2_splitq_rx_buf_desc *)(rxq->rx_ring))[i];
+ rxd->qword0.buf_id = i;
+ rxd->qword0.rsvd0 = 0;
+ rxd->qword0.rsvd1 = 0;
+ rxd->pkt_addr = dma_addr;
+ rxd->hdr_addr = 0;
+ rxd->rsvd2 = 0;
+
+ rxq->sw_ring[i] = mbuf;
+ }
+
+ rxq->nb_rx_hold = 0;
+ rxq->rx_tail = rxq->nb_rx_desc - 1;
+
+ return 0;
+}
+
+int
+idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+
+ if (rxq == NULL || !rxq->q_set) {
+ PMD_DRV_LOG(ERR, "RX queue %u not available or setup",
+ rx_queue_id);
+ return -EINVAL;
+ }
+
+ if (rxq->bufq1 == NULL) {
+ /* Single queue */
+ err = idpf_alloc_single_rxq_mbufs(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rxq->nb_rx_desc - 1);
+ } else {
+ /* Split queue */
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq1);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+ err = idpf_alloc_split_rxq_mbufs(rxq->bufq2);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate RX buffer queue mbuf");
+ return err;
+ }
+
+ rte_wmb();
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(rxq->bufq1->qrx_tail, rxq->bufq1->rx_tail);
+ IDPF_PCI_REG_WRITE(rxq->bufq2->qrx_tail, rxq->bufq2->rx_tail);
+ }
+
+ return err;
+}
+
+int
+idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq =
+ dev->data->rx_queues[rx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_rxq(vport, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Rx queue %u", rx_queue_id);
+ return err;
+ }
+
+ err = idpf_rx_queue_init(dev, rx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init RX queue %u",
+ rx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, rx_queue_id, true, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u on",
+ rx_queue_id);
+ } else {
+ rxq->q_started = true;
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
+
+int
+idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_tx_queue *txq;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ txq = dev->data->tx_queues[tx_queue_id];
+
+ /* Init the RX tail register. */
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, 0);
+
+ return 0;
+}
+
+int
+idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq =
+ dev->data->tx_queues[tx_queue_id];
+ int err = 0;
+
+ err = idpf_vc_config_txq(vport, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Fail to configure Tx queue %u", tx_queue_id);
+ return err;
+ }
+
+ err = idpf_tx_queue_init(dev, tx_queue_id);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to init TX queue %u",
+ tx_queue_id);
+ return err;
+ }
+
+ /* Ready to switch the queue on */
+ err = idpf_switch_queue(vport, tx_queue_id, false, true);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u on",
+ tx_queue_id);
+ } else {
+ txq->q_started = true;
+ dev->data->tx_queue_state[tx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STARTED;
+ }
+
+ return err;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3f3932c3eb..ab9b3830fd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -50,6 +50,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
+ bool rx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -95,6 +96,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
+ bool tx_deferred_start; /* don't start this queue in dev start */
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -109,8 +111,12 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index e89c2a2783..08fb9061c2 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -21,6 +21,7 @@
#include <rte_dev.h>
#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
static int
idpf_vc_clean(struct idpf_adapter *adapter)
@@ -223,6 +224,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
+ case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
+ case VIRTCHNL2_OP_ENABLE_QUEUES:
+ case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
/* for init virtchnl ops, need to poll the response */
@@ -390,6 +395,448 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+#define IDPF_RX_BUF_STRIDE 64
+int
+idpf_vc_config_rxqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i, j;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_rx_q + vport->num_rx_bufq;
+ while (total_qs) {
+ if (total_qs > adapter->max_rxq_per_msg) {
+ num_qs = adapter->max_rxq_per_msg;
+ total_qs -= adapter->max_rxq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 3; i++, k++) {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[i * 3];
+ rxq_info->dma_ring_addr =
+ rxq[k]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[k]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[k]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[k]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[k]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[k]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (j = 1; j <= IDPF_RX_BUFQ_PER_GRP; j++) {
+ struct idpf_rx_queue *bufq = j == 1 ?
+ rxq[k]->bufq1 : rxq[k]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i * 3 + j];
+ rxq_info->dma_ring_addr =
+ bufq->rx_ring_phys_addr;
+ rxq_info->type =
+ VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids =
+ VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride =
+ IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+ }
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_rx_queue **rxq =
+ (struct idpf_rx_queue **)vport->dev_data->rx_queues;
+ struct virtchnl2_config_rx_queues *vc_rxqs = NULL;
+ struct virtchnl2_rxq_info *rxq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err, i;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_RXQ_PER_GRP;
+ else
+ num_qs = IDPF_RXQ_PER_GRP + IDPF_RX_BUFQ_PER_GRP;
+
+ size = sizeof(*vc_rxqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_rxq_info);
+ vc_rxqs = rte_zmalloc("cfg_rxqs", size, 0);
+ if (vc_rxqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_rx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_rxqs->vport_id = vport->vport_id;
+ vc_rxqs->num_qinfo = num_qs;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SQ_NIC_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ } else {
+ /* Rx queue */
+ rxq_info = &vc_rxqs->qinfo[0];
+ rxq_info->dma_ring_addr = rxq[rxq_id]->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX;
+ rxq_info->queue_id = rxq[rxq_id]->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = rxq[rxq_id]->rx_buf_len;
+ rxq_info->max_pkt_size = vport->max_pkt_len;
+
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->qflags |= VIRTCHNL2_RX_DESC_SIZE_32BYTE;
+
+ rxq_info->ring_len = rxq[rxq_id]->nb_rx_desc;
+ rxq_info->rx_bufq1_id = rxq[rxq_id]->bufq1->queue_id;
+ rxq_info->rx_bufq2_id = rxq[rxq_id]->bufq2->queue_id;
+ rxq_info->rx_buffer_low_watermark = 64;
+
+ /* Buffer queue */
+ for (i = 1; i <= IDPF_RX_BUFQ_PER_GRP; i++) {
+ struct idpf_rx_queue *bufq =
+ i == 1 ? rxq[rxq_id]->bufq1 : rxq[rxq_id]->bufq2;
+ rxq_info = &vc_rxqs->qinfo[i];
+ rxq_info->dma_ring_addr = bufq->rx_ring_phys_addr;
+ rxq_info->type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ rxq_info->queue_id = bufq->queue_id;
+ rxq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ rxq_info->data_buffer_size = bufq->rx_buf_len;
+ rxq_info->desc_ids = VIRTCHNL2_RXDID_2_FLEX_SPLITQ_M;
+ rxq_info->ring_len = bufq->nb_rx_desc;
+
+ rxq_info->buffer_notif_stride = IDPF_RX_BUF_STRIDE;
+ rxq_info->rx_buffer_low_watermark = 64;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_RX_QUEUES;
+ args.in_args = (uint8_t *)vc_rxqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_rxqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_RX_QUEUES");
+
+ return err;
+}
+
+int
+idpf_vc_config_txqs(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t total_qs, num_qs;
+ int size, i;
+ int err = 0;
+ int k = 0;
+
+ total_qs = vport->num_tx_q + vport->num_tx_complq;
+ while (total_qs) {
+ if (total_qs > adapter->max_txq_per_msg) {
+ num_qs = adapter->max_txq_per_msg;
+ total_qs -= adapter->max_txq_per_msg;
+ } else {
+ num_qs = total_qs;
+ total_qs = 0;
+ }
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ break;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ for (i = 0; i < num_qs; i++, k++) {
+ txq_info = &vc_txqs->qinfo[i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ }
+ } else {
+ for (i = 0; i < num_qs / 2; i++, k++) {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[2 * i];
+ txq_info->dma_ring_addr = txq[k]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[k]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->nb_tx_desc;
+ txq_info->tx_compl_queue_id =
+ txq[k]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[2 * i + 1];
+ txq_info->dma_ring_addr =
+ txq[k]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[k]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[k]->complq->nb_tx_desc;
+ }
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+ break;
+ }
+ }
+
+ return err;
+}
+
+int
+idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct idpf_tx_queue **txq =
+ (struct idpf_tx_queue **)vport->dev_data->tx_queues;
+ struct virtchnl2_config_tx_queues *vc_txqs = NULL;
+ struct virtchnl2_txq_info *txq_info;
+ struct idpf_cmd_info args;
+ uint16_t num_qs;
+ int size, err;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE)
+ num_qs = IDPF_TXQ_PER_GRP;
+ else
+ num_qs = IDPF_TXQ_PER_GRP + IDPF_TX_COMPLQ_PER_GRP;
+
+ size = sizeof(*vc_txqs) + (num_qs - 1) *
+ sizeof(struct virtchnl2_txq_info);
+ vc_txqs = rte_zmalloc("cfg_txqs", size, 0);
+ if (vc_txqs == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate virtchnl2_config_tx_queues");
+ err = -ENOMEM;
+ return err;
+ }
+ vc_txqs->vport_id = vport->vport_id;
+ vc_txqs->num_qinfo = num_qs;
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SINGLE;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_QUEUE;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ } else {
+ /* txq info */
+ txq_info = &vc_txqs->qinfo[0];
+ txq_info->dma_ring_addr = txq[txq_id]->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX;
+ txq_info->queue_id = txq[txq_id]->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->nb_tx_desc;
+ txq_info->tx_compl_queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->relative_queue_id = txq_info->queue_id;
+
+ /* tx completion queue info */
+ txq_info = &vc_txqs->qinfo[1];
+ txq_info->dma_ring_addr = txq[txq_id]->complq->tx_ring_phys_addr;
+ txq_info->type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ txq_info->queue_id = txq[txq_id]->complq->queue_id;
+ txq_info->model = VIRTCHNL2_QUEUE_MODEL_SPLIT;
+ txq_info->sched_mode = VIRTCHNL2_TXQ_SCHED_MODE_FLOW;
+ txq_info->ring_len = txq[txq_id]->complq->nb_tx_desc;
+ }
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_CONFIG_TX_QUEUES;
+ args.in_args = (uint8_t *)vc_txqs;
+ args.in_args_size = size;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ rte_free(vc_txqs);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_CONFIG_TX_QUEUES");
+
+ return err;
+}
+
+static int
+idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
+ uint32_t type, bool on)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = 1;
+ queue_select->vport_id = vport->vport_id;
+
+ queue_chunk->type = type;
+ queue_chunk->start_queue_id = qid;
+ queue_chunk->num_queues = 1;
+
+ args.ops = on ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ on ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
+int
+idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
+ bool rx, bool on)
+{
+ uint32_t type;
+ int err, queue_id;
+
+ /* switch txq/rxq */
+ type = rx ? VIRTCHNL2_QUEUE_TYPE_RX : VIRTCHNL2_QUEUE_TYPE_TX;
+
+ if (type == VIRTCHNL2_QUEUE_TYPE_RX)
+ queue_id = vport->chunks_info.rx_start_qid + qid;
+ else
+ queue_id = vport->chunks_info.tx_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+
+ /* switch tx completion queue */
+ if (!rx && vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_id = vport->chunks_info.tx_compl_start_qid + qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ /* switch rx buffer queue */
+ if (rx && vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_id = vport->chunks_info.rx_buf_start_qid + 2 * qid;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ queue_id++;
+ err = idpf_vc_ena_dis_one_queue(vport, queue_id, type, on);
+ if (err != 0)
+ return err;
+ }
+
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 07/18] net/idpf: add support for queue stop
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (5 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 06/18] net/idpf: add support for queue start beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 08/18] net/idpf: add queue release beilei.xing
` (11 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for these device ops:
- rx_queue_stop
- tx_queue_stop
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 17 ++--
drivers/net/idpf/idpf_rxtx.c | 148 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 13 +++
drivers/net/idpf/idpf_vchnl.c | 69 +++++++++++++++
4 files changed, 242 insertions(+), 5 deletions(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 0400ed611f..9f1e1e6a18 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -324,7 +324,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
if (dev->data->mtu > vport->max_mtu) {
PMD_DRV_LOG(ERR, "MTU should be less than %d", vport->max_mtu);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_mtu;
}
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
@@ -332,17 +333,21 @@ idpf_dev_start(struct rte_eth_dev *dev)
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
- return ret;
+ goto err_mtu;
}
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
- /* TODO: stop queues */
- return ret;
+ goto err_vport;
}
return 0;
+
+err_vport:
+ idpf_stop_queues(dev);
+err_mtu:
+ return ret;
}
static int
@@ -352,7 +357,7 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_vc_ena_dis_vport(vport, false);
- /* TODO: stop queues */
+ idpf_stop_queues(dev);
return 0;
}
@@ -751,6 +756,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.link_update = idpf_dev_link_update,
.rx_queue_start = idpf_rx_queue_start,
.tx_queue_start = idpf_tx_queue_start,
+ .rx_queue_stop = idpf_rx_queue_stop,
+ .tx_queue_stop = idpf_tx_queue_stop,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 6d954afd9d..8d5ec41a1f 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -71,6 +71,55 @@ check_tx_thresh(uint16_t nb_desc, uint16_t tx_rs_thresh,
return 0;
}
+static void
+release_rxq_mbufs(struct idpf_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL)
+ return;
+
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL) {
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ rxq->sw_ring[i] = NULL;
+ }
+ }
+}
+
+static void
+release_txq_mbufs(struct idpf_tx_queue *txq)
+{
+ uint16_t nb_desc, i;
+
+ if (txq == NULL || txq->sw_ring == NULL) {
+ PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
+ return;
+ }
+
+ if (txq->sw_nb_desc != 0) {
+ /* For split queue model, descriptor ring */
+ nb_desc = txq->sw_nb_desc;
+ } else {
+ /* For single queue model */
+ nb_desc = txq->nb_tx_desc;
+ }
+ for (i = 0; i < nb_desc; i++) {
+ if (txq->sw_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+ txq->sw_ring[i].mbuf = NULL;
+ }
+ }
+}
+
+static const struct idpf_rxq_ops def_rxq_ops = {
+ .release_mbufs = release_rxq_mbufs,
+};
+
+static const struct idpf_txq_ops def_txq_ops = {
+ .release_mbufs = release_txq_mbufs,
+};
+
static void
reset_split_rx_descq(struct idpf_rx_queue *rxq)
{
@@ -122,6 +171,14 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static inline void
+reset_split_rx_queue(struct idpf_rx_queue *rxq)
+{
+ reset_split_rx_descq(rxq);
+ reset_split_rx_bufq(rxq->bufq1);
+ reset_split_rx_bufq(rxq->bufq2);
+}
+
static void
reset_single_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -301,6 +358,7 @@ idpf_rx_split_bufq_setup(struct rte_eth_dev *dev, struct idpf_rx_queue *bufq,
bufq->q_set = true;
bufq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_buf_qtail_start +
queue_idx * vport->chunks_info.rx_buf_qtail_spacing);
+ bufq->ops = &def_rxq_ops;
/* TODO: allow bulk or vec */
@@ -527,6 +585,7 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->rx_queues[queue_idx] = rxq;
rxq->qrx_tail = hw->hw_addr + (vport->chunks_info.rx_qtail_start +
queue_idx * vport->chunks_info.rx_qtail_spacing);
+ rxq->ops = &def_rxq_ops;
return 0;
}
@@ -621,6 +680,7 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
reset_split_tx_descq(txq);
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
/* Allocate the TX completion queue data structure. */
txq->complq = rte_zmalloc_socket("idpf splitq cq",
@@ -748,6 +808,7 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
dev->data->tx_queues[queue_idx] = txq;
txq->qtx_tail = hw->hw_addr + (vport->chunks_info.tx_qtail_start +
queue_idx * vport->chunks_info.tx_qtail_spacing);
+ txq->ops = &def_txq_ops;
return 0;
}
@@ -979,3 +1040,90 @@ idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return err;
}
+
+int
+idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_rx_queue *rxq;
+ int err;
+
+ if (rx_queue_id >= dev->data->nb_rx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, rx_queue_id, true, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch RX queue %u off",
+ rx_queue_id);
+ return err;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ rxq->ops->release_mbufs(rxq);
+ reset_single_rx_queue(rxq);
+ } else {
+ rxq->bufq1->ops->release_mbufs(rxq->bufq1);
+ rxq->bufq2->ops->release_mbufs(rxq->bufq2);
+ reset_split_rx_queue(rxq);
+ }
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+int
+idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_tx_queue *txq;
+ int err;
+
+ if (tx_queue_id >= dev->data->nb_tx_queues)
+ return -EINVAL;
+
+ err = idpf_switch_queue(vport, tx_queue_id, false, false);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "Failed to switch TX queue %u off",
+ tx_queue_id);
+ return err;
+ }
+
+ txq = dev->data->tx_queues[tx_queue_id];
+ txq->ops->release_mbufs(txq);
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SINGLE) {
+ reset_single_tx_queue(txq);
+ } else {
+ reset_split_tx_descq(txq);
+ reset_split_tx_complq(txq->complq);
+ }
+ dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+ return 0;
+}
+
+void
+idpf_stop_queues(struct rte_eth_dev *dev)
+{
+ struct idpf_rx_queue *rxq;
+ struct idpf_tx_queue *txq;
+ int i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (idpf_rx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Rx queue %d", i);
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+
+ if (idpf_tx_queue_stop(dev, i) != 0)
+ PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index ab9b3830fd..b959638489 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -51,6 +51,7 @@ struct idpf_rx_queue {
bool q_set; /* if rx queue has been configured */
bool q_started; /* if rx queue has been started */
bool rx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_rxq_ops *ops;
/* only valid for split queue mode */
uint8_t expected_gen_id;
@@ -97,6 +98,7 @@ struct idpf_tx_queue {
bool q_set; /* if tx queue has been configured */
bool q_started; /* if tx queue has been started */
bool tx_deferred_start; /* don't start this queue in dev start */
+ const struct idpf_txq_ops *ops;
/* only valid for split queue mode */
uint16_t sw_nb_desc;
@@ -107,16 +109,27 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+struct idpf_rxq_ops {
+ void (*release_mbufs)(struct idpf_rx_queue *rxq);
+};
+
+struct idpf_txq_ops {
+ void (*release_mbufs)(struct idpf_tx_queue *txq);
+};
+
int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 08fb9061c2..f92141b54f 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -837,6 +837,75 @@ idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
return err;
}
+#define IDPF_RXTX_QUEUE_CHUNKS_NUM 2
+int
+idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_del_ena_dis_queues *queue_select;
+ struct virtchnl2_queue_chunk *queue_chunk;
+ uint32_t type;
+ struct idpf_cmd_info args;
+ uint16_t num_chunks;
+ int err, len;
+
+ num_chunks = IDPF_RXTX_QUEUE_CHUNKS_NUM;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ num_chunks++;
+
+ len = sizeof(struct virtchnl2_del_ena_dis_queues) +
+ sizeof(struct virtchnl2_queue_chunk) * (num_chunks - 1);
+ queue_select = rte_zmalloc("queue_select", len, 0);
+ if (queue_select == NULL)
+ return -ENOMEM;
+
+ queue_chunk = queue_select->chunks.chunks;
+ queue_select->chunks.num_chunks = num_chunks;
+ queue_select->vport_id = vport->vport_id;
+
+ type = VIRTCHNL_QUEUE_TYPE_RX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.rx_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_q;
+
+ type = VIRTCHNL2_QUEUE_TYPE_TX;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id = vport->chunks_info.tx_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_q;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_RX_BUFFER;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.rx_buf_start_qid;
+ queue_chunk[type].num_queues = vport->num_rx_bufq;
+ }
+
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ type = VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION;
+ queue_chunk[type].type = type;
+ queue_chunk[type].start_queue_id =
+ vport->chunks_info.tx_compl_start_qid;
+ queue_chunk[type].num_queues = vport->num_tx_complq;
+ }
+
+ args.ops = enable ? VIRTCHNL2_OP_ENABLE_QUEUES :
+ VIRTCHNL2_OP_DISABLE_QUEUES;
+ args.in_args = (u8 *)queue_select;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUES",
+ enable ? "ENABLE" : "DISABLE");
+
+ rte_free(queue_select);
+ return err;
+}
+
int
idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
{
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 08/18] net/idpf: add queue release
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (6 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 07/18] net/idpf: add support for queue stop beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 09/18] net/idpf: add support for MTU configuration beilei.xing
` (10 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for queue operations:
- rx_queue_release
- tx_queue_release
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 81 ++++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 3 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 9f1e1e6a18..1485f40e71 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -758,6 +758,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_start = idpf_tx_queue_start,
.rx_queue_stop = idpf_rx_queue_stop,
.tx_queue_stop = idpf_tx_queue_stop,
+ .rx_queue_release = idpf_dev_rx_queue_release,
+ .tx_queue_release = idpf_dev_tx_queue_release,
};
static uint16_t
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 8d5ec41a1f..053409b99a 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -171,6 +171,51 @@ reset_split_rx_bufq(struct idpf_rx_queue *rxq)
rxq->bufq2 = NULL;
}
+static void
+idpf_rx_queue_release(void *rxq)
+{
+ struct idpf_rx_queue *q = rxq;
+
+ if (q == NULL)
+ return;
+
+ /* Split queue */
+ if (q->bufq1 != NULL && q->bufq2 != NULL) {
+ q->bufq1->ops->release_mbufs(q->bufq1);
+ rte_free(q->bufq1->sw_ring);
+ rte_memzone_free(q->bufq1->mz);
+ rte_free(q->bufq1);
+ q->bufq2->ops->release_mbufs(q->bufq2);
+ rte_free(q->bufq2->sw_ring);
+ rte_memzone_free(q->bufq2->mz);
+ rte_free(q->bufq2);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+ return;
+ }
+
+ /* Single queue */
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
+static void
+idpf_tx_queue_release(void *txq)
+{
+ struct idpf_tx_queue *q = txq;
+
+ if (q == NULL)
+ return;
+
+ rte_free(q->complq);
+ q->ops->release_mbufs(q);
+ rte_free(q->sw_ring);
+ rte_memzone_free(q->mz);
+ rte_free(q);
+}
+
static inline void
reset_split_rx_queue(struct idpf_rx_queue *rxq)
{
@@ -392,6 +437,12 @@ idpf_rx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -524,6 +575,12 @@ idpf_rx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_rx_thresh(nb_desc, rx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ idpf_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
/* Setup Rx description queue */
rxq = rte_zmalloc_socket("idpf rxq",
sizeof(struct idpf_rx_queue),
@@ -630,6 +687,12 @@ idpf_tx_split_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf split txq",
sizeof(struct idpf_tx_queue),
@@ -754,6 +817,12 @@ idpf_tx_single_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
if (check_tx_thresh(nb_desc, tx_rs_thresh, tx_free_thresh) != 0)
return -EINVAL;
+ /* Free memory if needed. */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ idpf_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
/* Allocate the TX queue data structure. */
txq = rte_zmalloc_socket("idpf txq",
sizeof(struct idpf_tx_queue),
@@ -1102,6 +1171,18 @@ idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id)
return 0;
}
+void
+idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_rx_queue_release(dev->data->rx_queues[qid]);
+}
+
+void
+idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid)
+{
+ idpf_tx_queue_release(dev->data->tx_queues[qid]);
+}
+
void
idpf_stop_queues(struct rte_eth_dev *dev)
{
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index b959638489..37944edcac 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -124,12 +124,15 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
+void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
void idpf_stop_queues(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 09/18] net/idpf: add support for MTU configuration
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (7 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 08/18] net/idpf: add queue release beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 10/18] net/idpf: add support for basic Rx datapath beilei.xing
` (9 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Add dev ops mtu_set.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 13 +++++++++++++
2 files changed, 14 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 46aab2eb61..d722c49fde 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -4,6 +4,7 @@
; Refer to default.ini for the full list of available PMD features.
;
[Features]
+MTU update = Y
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 1485f40e71..856f3d7266 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -83,6 +83,18 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return 0;
}
+static int
+idpf_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu __rte_unused)
+{
+ /* mtu setting is forbidden if port is start */
+ if (dev->data->dev_started) {
+ PMD_DRV_LOG(ERR, "port must be stopped before configuration");
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
static int
idpf_init_vport_req_info(struct rte_eth_dev *dev)
{
@@ -760,6 +772,7 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
.tx_queue_stop = idpf_tx_queue_stop,
.rx_queue_release = idpf_dev_rx_queue_release,
.tx_queue_release = idpf_dev_tx_queue_release,
+ .mtu_set = idpf_dev_mtu_set,
};
static uint16_t
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 10/18] net/idpf: add support for basic Rx datapath
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (8 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 09/18] net/idpf: add support for MTU configuration beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 11/18] net/idpf: add support for basic Tx datapath beilei.xing
` (8 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Rx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 2 +
drivers/net/idpf/idpf_rxtx.c | 273 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 7 +-
3 files changed, 281 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 856f3d7266..2f1f9eeee5 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -348,6 +348,8 @@ idpf_dev_start(struct rte_eth_dev *dev)
goto err_mtu;
}
+ idpf_set_rx_function(dev);
+
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to enable vport");
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 053409b99a..ea499c4d37 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1208,3 +1208,276 @@ idpf_stop_queues(struct rte_eth_dev *dev)
PMD_DRV_LOG(WARNING, "Fail to stop Tx queue %d", i);
}
}
+
+static void
+idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
+{
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_ring;
+ volatile struct virtchnl2_splitq_rx_buf_desc *rx_buf_desc;
+ uint16_t nb_refill = rx_bufq->rx_free_thresh;
+ uint16_t nb_desc = rx_bufq->nb_rx_desc;
+ uint16_t next_avail = rx_bufq->rx_tail;
+ struct rte_mbuf *nmb[rx_bufq->rx_free_thresh];
+ struct rte_eth_dev *dev;
+ uint64_t dma_addr;
+ uint16_t delta;
+ int i;
+
+ if (rx_bufq->nb_rx_hold < rx_bufq->rx_free_thresh)
+ return;
+
+ rx_buf_ring = rx_bufq->rx_ring;
+ delta = nb_desc - next_avail;
+ if (unlikely(delta < nb_refill)) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, delta) == 0)) {
+ for (i = 0; i < delta; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ nb_refill -= delta;
+ next_avail = 0;
+ rx_bufq->nb_rx_hold -= delta;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ return;
+ }
+ }
+
+ if (nb_desc - next_avail >= nb_refill) {
+ if (likely(rte_pktmbuf_alloc_bulk(rx_bufq->mp, nmb, nb_refill) == 0)) {
+ for (i = 0; i < nb_refill; i++) {
+ rx_buf_desc = &rx_buf_ring[next_avail + i];
+ rx_bufq->sw_ring[next_avail + i] = nmb[i];
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb[i]));
+ rx_buf_desc->hdr_addr = 0;
+ rx_buf_desc->pkt_addr = dma_addr;
+ }
+ next_avail += nb_refill;
+ rx_bufq->nb_rx_hold -= nb_refill;
+ } else {
+ dev = &rte_eth_devices[rx_bufq->port_id];
+ dev->data->rx_mbuf_alloc_failed += nb_desc - next_avail;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u queue_id=%u",
+ rx_bufq->port_id, rx_bufq->queue_id);
+ }
+ }
+
+ IDPF_PCI_REG_WRITE(rx_bufq->qrx_tail, next_avail);
+
+ rx_bufq->rx_tail = next_avail;
+}
+
+uint16_t
+idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc_ring;
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
+ uint16_t pktlen_gen_bufq_id;
+ struct idpf_rx_queue *rxq;
+ struct rte_mbuf *rxm;
+ uint16_t rx_id_bufq1;
+ uint16_t rx_id_bufq2;
+ uint16_t pkt_len;
+ uint16_t bufq_id;
+ uint16_t gen_id;
+ uint16_t rx_id;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_id_bufq1 = rxq->bufq1->rx_next_avail;
+ rx_id_bufq2 = rxq->bufq2->rx_next_avail;
+ rx_desc_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rx_desc = &rx_desc_ring[rx_id];
+
+ pktlen_gen_bufq_id =
+ rte_le_to_cpu_16(rx_desc->pktlen_gen_bufq_id);
+ gen_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S;
+ if (gen_id != rxq->expected_gen_id)
+ break;
+
+ pkt_len = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_LEN_PBUF_S;
+ if (pkt_len == 0)
+ PMD_RX_LOG(ERR, "Packet length is 0");
+
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc)) {
+ rx_id = 0;
+ rxq->expected_gen_id ^= 1;
+ }
+
+ bufq_id = (pktlen_gen_bufq_id &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_BUFQ_ID_S;
+ if (bufq_id == 0) {
+ rxm = rxq->bufq1->sw_ring[rx_id_bufq1];
+ rx_id_bufq1++;
+ if (unlikely(rx_id_bufq1 == rxq->bufq1->nb_rx_desc))
+ rx_id_bufq1 = 0;
+ rxq->bufq1->nb_rx_hold++;
+ } else {
+ rxm = rxq->bufq2->sw_ring[rx_id_bufq2];
+ rx_id_bufq2++;
+ if (unlikely(rx_id_bufq2 == rxq->bufq2->nb_rx_desc))
+ rx_id_bufq2 = 0;
+ rxq->bufq2->nb_rx_hold++;
+ }
+
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->next = NULL;
+ rxm->nb_segs = 1;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+
+ if (nb_rx > 0) {
+ rxq->rx_tail = rx_id;
+ if (rx_id_bufq1 != rxq->bufq1->rx_next_avail)
+ rxq->bufq1->rx_next_avail = rx_id_bufq1;
+ if (rx_id_bufq2 != rxq->bufq2->rx_next_avail)
+ rxq->bufq2->rx_next_avail = rx_id_bufq2;
+
+ idpf_split_rx_bufq_refill(rxq->bufq1);
+ idpf_split_rx_bufq_refill(rxq->bufq2);
+ }
+
+ return nb_rx;
+}
+
+static inline void
+idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
+ uint16_t rx_id)
+{
+ nb_hold = (uint16_t)(nb_hold + rxq->nb_rx_hold);
+
+ if (nb_hold > rxq->rx_free_thresh) {
+ PMD_RX_LOG(DEBUG,
+ "port_id=%u queue_id=%u rx_tail=%u nb_hold=%u",
+ rxq->port_id, rxq->queue_id, rx_id, nb_hold);
+ rx_id = (uint16_t)((rx_id == 0) ?
+ (rxq->nb_rx_desc - 1) : (rx_id - 1));
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+ nb_hold = 0;
+ }
+ rxq->nb_rx_hold = nb_hold;
+}
+
+uint16_t
+idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile union virtchnl2_rx_desc *rx_ring;
+ volatile union virtchnl2_rx_desc *rxdp;
+ union virtchnl2_rx_desc rxd;
+ struct idpf_rx_queue *rxq;
+ uint16_t rx_id, nb_hold;
+ struct rte_eth_dev *dev;
+ uint16_t rx_packet_len;
+ struct rte_mbuf *rxm;
+ struct rte_mbuf *nmb;
+ uint16_t rx_status0;
+ uint64_t dma_addr;
+ uint16_t nb_rx;
+
+ nb_rx = 0;
+ nb_hold = 0;
+ rxq = rx_queue;
+
+ if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
+ return nb_rx;
+
+ rx_id = rxq->rx_tail;
+ rx_ring = rxq->rx_ring;
+
+ while (nb_rx < nb_pkts) {
+ rxdp = &rx_ring[rx_id];
+ rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
+
+ /* Check the DD bit first */
+ if ((rx_status0 & (1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ break;
+
+ nmb = rte_mbuf_raw_alloc(rxq->mp);
+ if (unlikely(nmb == NULL)) {
+ dev = &rte_eth_devices[rxq->port_id];
+ dev->data->rx_mbuf_alloc_failed++;
+ PMD_RX_LOG(DEBUG, "RX mbuf alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+ rxd = *rxdp; /* copy descriptor in ring to temp variable*/
+
+ nb_hold++;
+ rxm = rxq->sw_ring[rx_id];
+ rxq->sw_ring[rx_id] = nmb;
+ rx_id++;
+ if (unlikely(rx_id == rxq->nb_rx_desc))
+ rx_id = 0;
+
+ /* Prefetch next mbuf */
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+
+ /* When next RX descriptor is on a cache line boundary,
+ * prefetch the next 4 RX descriptors and next 8 pointers
+ * to mbufs.
+ */
+ if ((rx_id & 0x3) == 0) {
+ rte_prefetch0(&rx_ring[rx_id]);
+ rte_prefetch0(rxq->sw_ring[rx_id]);
+ }
+ dma_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(nmb));
+ rxdp->read.hdr_addr = 0;
+ rxdp->read.pkt_addr = dma_addr;
+
+ rx_packet_len = (rte_cpu_to_le_16(rxd.flex_nic_wb.pkt_len) &
+ VIRTCHNL2_RX_FLEX_DESC_PKT_LEN_M);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rte_prefetch0(RTE_PTR_ADD(rxm->buf_addr, RTE_PKTMBUF_HEADROOM));
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->pkt_len = rx_packet_len;
+ rxm->data_len = rx_packet_len;
+ rxm->port = rxq->port_id;
+
+ rx_pkts[nb_rx++] = rxm;
+ }
+ rxq->rx_tail = rx_id;
+
+ idpf_update_rx_tail(rxq, nb_hold, rx_id);
+
+ return nb_rx;
+}
+
+void
+idpf_set_rx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ else
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 37944edcac..650c6c1c3a 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -133,6 +133,11 @@ int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
-
+uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
void idpf_stop_queues(struct rte_eth_dev *dev);
+
+void idpf_set_rx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 11/18] net/idpf: add support for basic Tx datapath
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (9 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 10/18] net/idpf: add support for basic Rx datapath beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 12/18] net/idpf: support parsing packet type beilei.xing
` (7 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add basic Tx support in split queue mode and single queue mode.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 3 +
drivers/net/idpf/idpf_ethdev.h | 1 +
drivers/net/idpf/idpf_rxtx.c | 357 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 10 +
4 files changed, 371 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 2f1f9eeee5..f9f6fe1162 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
.tx_rs_thresh = IDPF_DEFAULT_TX_RS_THRESH,
@@ -349,6 +351,7 @@ idpf_dev_start(struct rte_eth_dev *dev)
}
idpf_set_rx_function(dev);
+ idpf_set_tx_function(dev);
ret = idpf_vc_ena_dis_vport(vport, true);
if (ret != 0) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 96c22009e9..af0a8e2970 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -35,6 +35,7 @@
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
+#define IDPF_MIN_FRAME_SIZE 14
#define IDPF_NUM_MACADDR_MAX 64
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index ea499c4d37..f55d2143b9 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1365,6 +1365,148 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline void
+idpf_split_tx_free(struct idpf_tx_queue *cq)
+{
+ volatile struct idpf_splitq_tx_compl_desc *compl_ring = cq->compl_ring;
+ volatile struct idpf_splitq_tx_compl_desc *txd;
+ uint16_t next = cq->tx_tail;
+ struct idpf_tx_entry *txe;
+ struct idpf_tx_queue *txq;
+ uint16_t gen, qid, q_head;
+ uint8_t ctype;
+
+ txd = &compl_ring[next];
+ gen = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_GEN_M) >> IDPF_TXD_COMPLQ_GEN_S;
+ if (gen != cq->expected_gen_id)
+ return;
+
+ ctype = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_COMPL_TYPE_M) >> IDPF_TXD_COMPLQ_COMPL_TYPE_S;
+ qid = (rte_le_to_cpu_16(txd->qid_comptype_gen) &
+ IDPF_TXD_COMPLQ_QID_M) >> IDPF_TXD_COMPLQ_QID_S;
+ q_head = rte_le_to_cpu_16(txd->q_head_compl_tag.compl_tag);
+ txq = cq->txqs[qid - cq->tx_start_qid];
+
+ switch (ctype) {
+ case IDPF_TXD_COMPLT_RE:
+ if (q_head == 0)
+ txq->last_desc_cleaned = txq->nb_tx_desc - 1;
+ else
+ txq->last_desc_cleaned = q_head - 1;
+ if (unlikely((txq->last_desc_cleaned % 32) == 0)) {
+ PMD_DRV_LOG(ERR, "unexpected desc (head = %u) completion.",
+ q_head);
+ return;
+ }
+
+ break;
+ case IDPF_TXD_COMPLT_RS:
+ txq->nb_free++;
+ txq->nb_used--;
+ txe = &txq->sw_ring[q_head];
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+ break;
+ default:
+ PMD_DRV_LOG(ERR, "unknown completion type.");
+ return;
+ }
+
+ if (++next == cq->nb_tx_desc) {
+ next = 0;
+ cq->expected_gen_id ^= 1;
+ }
+
+ cq->tx_tail = next;
+}
+
+uint16_t
+idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = (struct idpf_tx_queue *)tx_queue;
+ volatile struct idpf_flex_tx_sched_desc *txr;
+ volatile struct idpf_flex_tx_sched_desc *txd;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_entry *txe, *txn;
+ uint16_t nb_used, tx_id, sw_id;
+ struct rte_mbuf *tx_pkt;
+ uint16_t nb_to_clean;
+ uint16_t nb_tx = 0;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ txr = txq->desc_ring;
+ sw_ring = txq->sw_ring;
+ tx_id = txq->tx_tail;
+ sw_id = txq->sw_tail;
+ txe = &sw_ring[sw_id];
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ tx_pkt = tx_pkts[nb_tx];
+
+ if (txq->nb_free <= txq->free_thresh) {
+ /* TODO: Need to refine
+ * 1. free and clean: Better to decide a clean destination instead of
+ * loop times. And don't free mbuf when RS got immediately, free when
+ * transmit or according to the clean destination.
+ * Now, just ignore the RE write back, free mbuf when get RS
+ * 2. out-of-order rewrite back haven't be supported, SW head and HW head
+ * need to be separated.
+ **/
+ nb_to_clean = 2 * txq->rs_thresh;
+ while (nb_to_clean--)
+ idpf_split_tx_free(txq->complq);
+ }
+
+ if (txq->nb_free < tx_pkt->nb_segs)
+ break;
+ nb_used = tx_pkt->nb_segs;
+
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+ txe->mbuf = tx_pkt;
+
+ /* Setup TX descriptor */
+ txd->buf_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova(tx_pkt));
+ txd->qw1.cmd_dtype =
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE);
+ txd->qw1.rxr_bufsize = tx_pkt->data_len;
+ txd->qw1.compl_tag = sw_id;
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ sw_id = txe->next_id;
+ txe = txn;
+ tx_pkt = tx_pkt->next;
+ } while (tx_pkt);
+
+ /* fill the last descriptor with End of Packet (EOP) bit */
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_EOP;
+
+ if (unlikely((tx_id % 32) == 0))
+ txd->qw1.cmd_dtype |= IDPF_TXD_FLEX_FLOW_CMD_RE;
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ }
+
+ /* update the tail pointer if any packets were processed */
+ if (likely(nb_tx > 0)) {
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+ txq->sw_tail = sw_id;
+ }
+
+ return nb_tx;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1471,6 +1613,208 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
return nb_rx;
}
+static inline int
+idpf_xmit_cleanup(struct idpf_tx_queue *txq)
+{
+ uint16_t last_desc_cleaned = txq->last_desc_cleaned;
+ struct idpf_tx_entry *sw_ring = txq->sw_ring;
+ uint16_t nb_tx_desc = txq->nb_tx_desc;
+ uint16_t desc_to_clean_to;
+ uint16_t nb_tx_to_clean;
+ uint16_t i;
+
+ volatile struct idpf_flex_tx_desc *txd = txq->tx_ring;
+
+ desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->rs_thresh);
+ if (desc_to_clean_to >= nb_tx_desc)
+ desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc);
+
+ desc_to_clean_to = sw_ring[desc_to_clean_to].last_id;
+ /* In the writeback Tx desccriptor, the only significant fields are the 4-bit DTYPE */
+ if ((txd[desc_to_clean_to].qw1.cmd_dtype &
+ rte_cpu_to_le_16(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_DESC_DONE)) {
+ PMD_TX_LOG(DEBUG, "TX descriptor %4u is not done "
+ "(port=%d queue=%d)", desc_to_clean_to,
+ txq->port_id, txq->queue_id);
+ return -1;
+ }
+
+ if (last_desc_cleaned > desc_to_clean_to)
+ nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) +
+ desc_to_clean_to);
+ else
+ nb_tx_to_clean = (uint16_t)(desc_to_clean_to -
+ last_desc_cleaned);
+
+ txd[desc_to_clean_to].qw1.cmd_dtype = 0;
+ txd[desc_to_clean_to].qw1.buf_size = 0;
+ for (i = 0; i < RTE_DIM(txd[desc_to_clean_to].qw1.flex.raw); i++)
+ txd[desc_to_clean_to].qw1.flex.raw[i] = 0;
+
+ txq->last_desc_cleaned = desc_to_clean_to;
+ txq->nb_free = (uint16_t)(txq->nb_free + nb_tx_to_clean);
+
+ return 0;
+}
+
+/* TX function */
+uint16_t
+idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ volatile struct idpf_flex_tx_desc *txd;
+ volatile struct idpf_flex_tx_desc *txr;
+ struct idpf_tx_entry *txe, *txn;
+ struct idpf_tx_entry *sw_ring;
+ struct idpf_tx_queue *txq;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ uint64_t buf_dma_addr;
+ uint16_t tx_last;
+ uint16_t nb_used;
+ uint16_t td_cmd;
+ uint16_t tx_id;
+ uint16_t nb_tx;
+ uint16_t slen;
+
+ nb_tx = 0;
+ txq = tx_queue;
+
+ if (unlikely(txq == NULL) || unlikely(!txq->q_started))
+ return nb_tx;
+
+ sw_ring = txq->sw_ring;
+ txr = txq->tx_ring;
+ tx_id = txq->tx_tail;
+ txe = &sw_ring[tx_id];
+
+ /* Check if the descriptor ring needs to be cleaned. */
+ if (txq->nb_free < txq->free_thresh)
+ (void)idpf_xmit_cleanup(txq);
+
+ for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
+ td_cmd = 0;
+
+ tx_pkt = *tx_pkts++;
+ RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+
+ /* The number of descriptors that must be allocated for
+ * a packet equals to the number of the segments of that
+ * packet plus 1 context descriptor if needed.
+ */
+ nb_used = (uint16_t)(tx_pkt->nb_segs);
+ tx_last = (uint16_t)(tx_id + nb_used - 1);
+
+ /* Circular ring */
+ if (tx_last >= txq->nb_tx_desc)
+ tx_last = (uint16_t)(tx_last - txq->nb_tx_desc);
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u"
+ " tx_first=%u tx_last=%u",
+ txq->port_id, txq->queue_id, tx_id, tx_last);
+
+ if (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ if (unlikely(nb_used > txq->rs_thresh)) {
+ while (nb_used > txq->nb_free) {
+ if (idpf_xmit_cleanup(txq) != 0) {
+ if (nb_tx == 0)
+ return 0;
+ goto end_of_tx;
+ }
+ }
+ }
+ }
+
+ m_seg = tx_pkt;
+ do {
+ txd = &txr[tx_id];
+ txn = &sw_ring[txe->next_id];
+
+ if (txe->mbuf != NULL)
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = m_seg;
+
+ /* Setup TX Descriptor */
+ slen = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ txd->qw1.buf_size = slen;
+ txd->qw1.cmd_dtype = rte_cpu_to_le_16(IDPF_TX_DESC_DTYPE_FLEX_DATA <<
+ IDPF_FLEX_TXD_QW1_DTYPE_S);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ /* The last packet data descriptor needs End Of Packet (EOP) */
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_EOP;
+ txq->nb_used = (uint16_t)(txq->nb_used + nb_used);
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_used);
+
+ if (txq->nb_used >= txq->rs_thresh) {
+ PMD_TX_LOG(DEBUG, "Setting RS bit on TXD id="
+ "%4u (port=%d queue=%d)",
+ tx_last, txq->port_id, txq->queue_id);
+
+ td_cmd |= IDPF_TX_FLEX_DESC_CMD_RS;
+
+ /* Update txq RS bit counters */
+ txq->nb_used = 0;
+ }
+
+ txd->qw1.cmd_dtype |= rte_cpu_to_le_16(td_cmd << IDPF_FLEX_TXD_QW1_CMD_S);
+ }
+
+end_of_tx:
+ rte_wmb();
+
+ PMD_TX_LOG(DEBUG, "port_id=%u queue_id=%u tx_tail=%u nb_tx=%u",
+ txq->port_id, txq->queue_id, tx_id, nb_tx);
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, tx_id);
+ txq->tx_tail = tx_id;
+
+ return nb_tx;
+}
+
+/* TX prep functions */
+uint16_t
+idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /* Check condition for nb_segs > IDPF_TX_MAX_MTU_SEG. */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) == 0) {
+ if (m->nb_segs > IDPF_TX_MAX_MTU_SEG) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
+ rte_errno = EINVAL;
+ return i;
+ }
+ }
+
+ return i;
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
@@ -1481,3 +1825,16 @@ idpf_set_rx_function(struct rte_eth_dev *dev)
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
}
+
+void
+idpf_set_tx_function(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ } else {
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ }
+}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 650c6c1c3a..30dc94b3dd 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -21,6 +21,8 @@
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
+#define IDPF_TX_MAX_MTU_SEG 10
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -137,7 +139,15 @@ uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
+void idpf_set_tx_function(struct rte_eth_dev *dev);
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 12/18] net/idpf: support parsing packet type
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (10 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 11/18] net/idpf: add support for basic Tx datapath beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
` (6 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Parse packet type during receiving packets.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 6 +
drivers/net/idpf/idpf_ethdev.h | 6 +
drivers/net/idpf/idpf_rxtx.c | 11 ++
drivers/net/idpf/idpf_rxtx.h | 5 +
drivers/net/idpf/idpf_vchnl.c | 240 +++++++++++++++++++++++++++++++++
5 files changed, 268 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index f9f6fe1162..d0821ec3f3 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -686,6 +686,12 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
goto err_api;
}
+ ret = idpf_get_pkt_type(adapter);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to set ptype table");
+ goto err_api;
+ }
+
adapter->caps = rte_zmalloc("idpf_caps",
sizeof(struct virtchnl2_get_capabilities), 0);
if (adapter->caps == NULL) {
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index af0a8e2970..db9af58f72 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -39,6 +39,8 @@
#define IDPF_NUM_MACADDR_MAX 64
+#define IDPF_MAX_PKT_TYPE 1024
+
#define IDPF_VLAN_TAG_SIZE 4
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
@@ -125,6 +127,8 @@ struct idpf_adapter {
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
+
+ uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
@@ -182,6 +186,7 @@ atomic_set_cmd(struct idpf_adapter *adapter, enum virtchnl_ops ops)
struct idpf_adapter *idpf_find_adapter(struct rte_pci_device *pci_dev);
void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev);
int idpf_vc_check_api_version(struct idpf_adapter *adapter);
+int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
@@ -193,6 +198,7 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f55d2143b9..a980714060 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1281,6 +1281,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc;
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1300,6 +1301,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id_bufq1 = rxq->bufq1->rx_next_avail;
rx_id_bufq2 = rxq->bufq2->rx_next_avail;
rx_desc_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1347,6 +1349,10 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
+ VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
rx_pkts[nb_rx++] = rxm;
}
@@ -1533,6 +1539,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
volatile union virtchnl2_rx_desc *rxdp;
union virtchnl2_rx_desc rxd;
struct idpf_rx_queue *rxq;
+ const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
uint16_t rx_packet_len;
@@ -1551,6 +1558,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_id = rxq->rx_tail;
rx_ring = rxq->rx_ring;
+ ptype_tbl = rxq->adapter->ptype_tbl;
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
@@ -1603,6 +1611,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->packet_type =
+ ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
+ VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
rx_pkts[nb_rx++] = rxm;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 30dc94b3dd..3853ed55c9 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,10 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_GET_PTYPE_SIZE(p) \
+ (sizeof(struct virtchnl2_ptype) + \
+ (((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -150,4 +154,5 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+
#endif /* _IDPF_RXTX_H_ */
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index f92141b54f..ced64dfaf6 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -234,6 +234,11 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
break;
+ case VIRTCHNL2_OP_GET_PTYPE_INFO:
+ /* for multuple response message,
+ * do not handle the response here.
+ */
+ break;
default:
/* For other virtchnl ops in running time,
* wait for the cmd done flag.
@@ -298,6 +303,215 @@ idpf_vc_check_api_version(struct idpf_adapter *adapter)
return 0;
}
+int __rte_cold
+idpf_get_pkt_type(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ uint16_t ptype_recvd = 0, ptype_offset, i, j;
+ int ret;
+
+ ret = idpf_vc_query_ptype_info(adapter);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to query packet type information");
+ return ret;
+ }
+
+ ptype_info = rte_zmalloc("ptype_info", IDPF_DFLT_MBX_BUF_SIZE, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ while (ptype_recvd < IDPF_MAX_PKT_TYPE) {
+ ret = idpf_read_one_msg(adapter, VIRTCHNL2_OP_GET_PTYPE_INFO,
+ IDPF_DFLT_MBX_BUF_SIZE, (u8 *)ptype_info);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Fail to get packet type information");
+ goto free_ptype_info;
+ }
+
+ ptype_recvd += ptype_info->num_ptypes;
+ ptype_offset = sizeof(struct virtchnl2_get_ptype_info) -
+ sizeof(struct virtchnl2_ptype);
+
+ for (i = 0; i < rte_cpu_to_le_16(ptype_info->num_ptypes); i++) {
+ bool is_inner = false, is_ip = false;
+ struct virtchnl2_ptype *ptype;
+ uint32_t proto_hdr = 0;
+
+ ptype = (struct virtchnl2_ptype *)
+ ((u8 *)ptype_info + ptype_offset);
+ ptype_offset += IDPF_GET_PTYPE_SIZE(ptype);
+ if (ptype_offset > IDPF_DFLT_MBX_BUF_SIZE) {
+ ret = -EINVAL;
+ goto free_ptype_info;
+ }
+
+ if (rte_cpu_to_le_16(ptype->ptype_id_10) == 0xFFFF)
+ goto free_ptype_info;
+
+ for (j = 0; j < ptype->proto_id_count; j++) {
+ switch (rte_cpu_to_le_16(ptype->proto_id[j])) {
+ case VIRTCHNL2_PROTO_HDR_GRE:
+ case VIRTCHNL2_PROTO_HDR_VXLAN:
+ proto_hdr &= ~RTE_PTYPE_L4_MASK;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GRENAT;
+ is_inner = true;
+ break;
+ case VIRTCHNL2_PROTO_HDR_MAC:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER;
+ } else {
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_VLAN:
+ if (is_inner) {
+ proto_hdr &= ~RTE_PTYPE_INNER_L2_MASK;
+ proto_hdr |= RTE_PTYPE_INNER_L2_ETHER_VLAN;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_PTP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_TIMESYNC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_LLDP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_LLDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ARP:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_ARP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PPPOE:
+ proto_hdr &= ~RTE_PTYPE_L2_MASK;
+ proto_hdr |= RTE_PTYPE_L2_ETHER_PPPOE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV4_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV6:
+ if (!is_ip) {
+ proto_hdr |= RTE_PTYPE_L3_IPV6_EXT_UNKNOWN;
+ is_ip = true;
+ } else {
+ proto_hdr |= RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+ RTE_PTYPE_TUNNEL_IP;
+ is_inner = true;
+ }
+ break;
+ case VIRTCHNL2_PROTO_HDR_IPV4_FRAG:
+ case VIRTCHNL2_PROTO_HDR_IPV6_FRAG:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_FRAG;
+ else
+ proto_hdr |= RTE_PTYPE_L4_FRAG;
+ break;
+ case VIRTCHNL2_PROTO_HDR_UDP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_UDP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_UDP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_TCP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_TCP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_TCP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_SCTP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_SCTP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_SCTP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMP:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_ICMPV6:
+ if (is_inner)
+ proto_hdr |= RTE_PTYPE_INNER_L4_ICMP;
+ else
+ proto_hdr |= RTE_PTYPE_L4_ICMP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_L2TPV2:
+ case VIRTCHNL2_PROTO_HDR_L2TPV2_CONTROL:
+ case VIRTCHNL2_PROTO_HDR_L2TPV3:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_L2TP;
+ break;
+ case VIRTCHNL2_PROTO_HDR_NVGRE:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_NVGRE;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPC_TEID:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPC;
+ break;
+ case VIRTCHNL2_PROTO_HDR_GTPU:
+ case VIRTCHNL2_PROTO_HDR_GTPU_UL:
+ case VIRTCHNL2_PROTO_HDR_GTPU_DL:
+ is_inner = true;
+ proto_hdr |= RTE_PTYPE_TUNNEL_GTPU;
+ break;
+ case VIRTCHNL2_PROTO_HDR_PAY:
+ case VIRTCHNL2_PROTO_HDR_IPV6_EH:
+ case VIRTCHNL2_PROTO_HDR_PRE_MAC:
+ case VIRTCHNL2_PROTO_HDR_POST_MAC:
+ case VIRTCHNL2_PROTO_HDR_ETHERTYPE:
+ case VIRTCHNL2_PROTO_HDR_SVLAN:
+ case VIRTCHNL2_PROTO_HDR_CVLAN:
+ case VIRTCHNL2_PROTO_HDR_MPLS:
+ case VIRTCHNL2_PROTO_HDR_MMPLS:
+ case VIRTCHNL2_PROTO_HDR_CTRL:
+ case VIRTCHNL2_PROTO_HDR_ECP:
+ case VIRTCHNL2_PROTO_HDR_EAPOL:
+ case VIRTCHNL2_PROTO_HDR_PPPOD:
+ case VIRTCHNL2_PROTO_HDR_IGMP:
+ case VIRTCHNL2_PROTO_HDR_AH:
+ case VIRTCHNL2_PROTO_HDR_ESP:
+ case VIRTCHNL2_PROTO_HDR_IKE:
+ case VIRTCHNL2_PROTO_HDR_NATT_KEEP:
+ case VIRTCHNL2_PROTO_HDR_GTP:
+ case VIRTCHNL2_PROTO_HDR_GTP_EH:
+ case VIRTCHNL2_PROTO_HDR_GTPCV2:
+ case VIRTCHNL2_PROTO_HDR_ECPRI:
+ case VIRTCHNL2_PROTO_HDR_VRRP:
+ case VIRTCHNL2_PROTO_HDR_OSPF:
+ case VIRTCHNL2_PROTO_HDR_TUN:
+ case VIRTCHNL2_PROTO_HDR_VXLAN_GPE:
+ case VIRTCHNL2_PROTO_HDR_GENEVE:
+ case VIRTCHNL2_PROTO_HDR_NSH:
+ case VIRTCHNL2_PROTO_HDR_QUIC:
+ case VIRTCHNL2_PROTO_HDR_PFCP:
+ case VIRTCHNL2_PROTO_HDR_PFCP_NODE:
+ case VIRTCHNL2_PROTO_HDR_PFCP_SESSION:
+ case VIRTCHNL2_PROTO_HDR_RTP:
+ case VIRTCHNL2_PROTO_HDR_NO_PROTO:
+ default:
+ continue;
+ }
+ adapter->ptype_tbl[ptype->ptype_id_10] = proto_hdr;
+ }
+ }
+ }
+
+free_ptype_info:
+ rte_free(ptype_info);
+ clear_cmd(adapter);
+ return ret;
+}
+
int
idpf_vc_get_caps(struct idpf_adapter *adapter)
{
@@ -930,3 +1144,29 @@ idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable)
return err;
}
+
+int
+idpf_vc_query_ptype_info(struct idpf_adapter *adapter)
+{
+ struct virtchnl2_get_ptype_info *ptype_info;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(struct virtchnl2_get_ptype_info);
+ ptype_info = rte_zmalloc("ptype_info", len, 0);
+ if (ptype_info == NULL)
+ return -ENOMEM;
+
+ ptype_info->start_ptype_id = 0;
+ ptype_info->num_ptypes = IDPF_MAX_PKT_TYPE;
+ args.ops = VIRTCHNL2_OP_GET_PTYPE_INFO;
+ args.in_args = (u8 *)ptype_info;
+ args.in_args_size = len;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
+
+ rte_free(ptype_info);
+ return err;
+}
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 13/18] net/idpf: add support for write back based on ITR expire
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (11 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 12/18] net/idpf: support parsing packet type beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 14/18] net/idpf: add support for RSS beilei.xing
` (5 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo
From: Junfeng Guo <junfeng.guo@intel.com>
Enable write back on ITR expire, then packets can be received one by
one.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 +++++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_ethdev.h | 13 ++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 246 insertions(+)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index d0821ec3f3..957cc10616 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -297,6 +297,90 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rx_queues_irqs(struct rte_eth_dev *dev)
+{
+ struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector *qv_map;
+ struct idpf_hw *hw = &adapter->hw;
+ uint32_t dynctl_reg_start;
+ uint32_t itrn_reg_start;
+ uint32_t dynctl_val, itrn_val;
+ uint16_t i;
+
+ qv_map = rte_zmalloc("qv_map",
+ dev->data->nb_rx_queues *
+ sizeof(struct virtchnl2_queue_vector), 0);
+ if (qv_map == NULL) {
+ PMD_DRV_LOG(ERR, "Failed to allocate %d queue-vector map",
+ dev->data->nb_rx_queues);
+ goto qv_map_alloc_err;
+ }
+
+ /* Rx interrupt disabled, Map interrupt only for writeback */
+
+ /* The capability flags adapter->caps->other_caps should be
+ * compared with bit VIRTCHNL2_CAP_WB_ON_ITR here. The if
+ * condition should be updated when the FW can return the
+ * correct flag bits.
+ */
+ dynctl_reg_start =
+ vport->recv_vectors->vchunks.vchunks->dynctl_reg_start;
+ itrn_reg_start =
+ vport->recv_vectors->vchunks.vchunks->itrn_reg_start;
+ dynctl_val = IDPF_READ_REG(hw, dynctl_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of dynctl_reg_start is 0x%x",
+ dynctl_val);
+ itrn_val = IDPF_READ_REG(hw, itrn_reg_start);
+ PMD_DRV_LOG(DEBUG, "Value of itrn_reg_start is 0x%x", itrn_val);
+ /* Force write-backs by setting WB_ON_ITR bit in DYN_CTL
+ * register. WB_ON_ITR and INTENA are mutually exclusive
+ * bits. Setting WB_ON_ITR bits means TX and RX Descs
+ * are written back based on ITR expiration irrespective
+ * of INTENA setting.
+ */
+ /* TBD: need to tune INTERVAL value for better performance. */
+ if (itrn_val != 0)
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ itrn_val <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+ else
+ IDPF_WRITE_REG(hw,
+ dynctl_reg_start,
+ VIRTCHNL2_ITR_IDX_0 <<
+ PF_GLINT_DYN_CTL_ITR_INDX_S |
+ PF_GLINT_DYN_CTL_WB_ON_ITR_M |
+ IDPF_DFLT_INTERVAL <<
+ PF_GLINT_DYN_CTL_INTERVAL_S);
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ /* map all queues to the same vector */
+ qv_map[i].queue_id = vport->chunks_info.rx_start_qid + i;
+ qv_map[i].vector_id =
+ vport->recv_vectors->vchunks.vchunks->start_vector_id;
+ }
+ vport->qv_map = qv_map;
+
+ if (idpf_vc_config_irq_map_unmap(vport, true) != 0) {
+ PMD_DRV_LOG(ERR, "config interrupt mapping failed");
+ goto config_irq_map_err;
+ }
+
+ return 0;
+
+config_irq_map_err:
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
+
+qv_map_alloc_err:
+ return -1;
+}
+
static int
idpf_start_queues(struct rte_eth_dev *dev)
{
@@ -334,6 +418,10 @@ static int
idpf_dev_start(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+ struct idpf_adapter *adapter = vport->adapter;
+ uint16_t num_allocated_vectors =
+ adapter->caps->num_allocated_vectors;
+ uint16_t req_vecs_num;
int ret;
if (dev->data->mtu > vport->max_mtu) {
@@ -344,6 +432,27 @@ idpf_dev_start(struct rte_eth_dev *dev)
vport->max_pkt_len = dev->data->mtu + IDPF_ETH_OVERHEAD;
+ req_vecs_num = IDPF_DFLT_Q_VEC_NUM;
+ if (req_vecs_num + adapter->used_vecs_num > num_allocated_vectors) {
+ PMD_DRV_LOG(ERR, "The accumulated request vectors' number should be less than %d",
+ num_allocated_vectors);
+ ret = -EINVAL;
+ goto err_mtu;
+ }
+
+ ret = idpf_vc_alloc_vectors(vport, req_vecs_num);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to allocate interrupt vectors");
+ goto err_mtu;
+ }
+ adapter->used_vecs_num += req_vecs_num;
+
+ ret = idpf_config_rx_queues_irqs(dev);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to configure irqs");
+ goto err_mtu;
+ }
+
ret = idpf_start_queues(dev);
if (ret != 0) {
PMD_DRV_LOG(ERR, "Failed to start queues");
@@ -376,6 +485,10 @@ idpf_dev_stop(struct rte_eth_dev *dev)
idpf_stop_queues(dev);
+ idpf_vc_config_irq_map_unmap(vport, false);
+
+ idpf_vc_dealloc_vectors(vport);
+
return 0;
}
@@ -387,6 +500,11 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->recv_vectors);
+ vport->recv_vectors = NULL;
+
+ rte_free(vport->qv_map);
+ vport->qv_map = NULL;
adapter->cur_vports &= ~RTE_BIT32(vport->devarg_id);
@@ -748,6 +866,8 @@ idpf_adapter_init(struct rte_pci_device *pci_dev, struct idpf_adapter *adapter)
adapter->cur_vports = 0;
adapter->cur_vport_nb = 0;
+ adapter->used_vecs_num = 0;
+
return ret;
err_vports:
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index db9af58f72..811240c386 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -33,6 +33,9 @@
#define IDPF_CTLQ_LEN 64
#define IDPF_DFLT_MBX_BUF_SIZE 4096
+#define IDPF_DFLT_Q_VEC_NUM 1
+#define IDPF_DFLT_INTERVAL 16
+
#define IDPF_MIN_BUF_SIZE 1024
#define IDPF_MAX_FRAME_SIZE 9728
#define IDPF_MIN_FRAME_SIZE 14
@@ -92,6 +95,11 @@ struct idpf_vport {
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* MSIX info*/
+ struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
+ uint16_t max_vectors;
+ struct virtchnl2_alloc_vectors *recv_vectors;
+
/* Chunk info */
struct idpf_chunks_info chunks_info;
@@ -124,6 +132,8 @@ struct idpf_adapter {
uint16_t cur_vport_nb;
uint16_t cur_vport_idx;
+ uint16_t used_vecs_num;
+
/* Max config queue number per VC message */
uint32_t max_rxq_per_msg;
uint32_t max_txq_per_msg;
@@ -198,6 +208,9 @@ int idpf_switch_queue(struct idpf_vport *vport, uint16_t qid,
bool rx, bool on);
int idpf_vc_ena_dis_queues(struct idpf_vport *vport, bool enable);
int idpf_vc_ena_dis_vport(struct idpf_vport *vport, bool enable);
+int idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map);
+int idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors);
+int idpf_vc_dealloc_vectors(struct idpf_vport *vport);
int idpf_vc_query_ptype_info(struct idpf_adapter *adapter);
int idpf_read_one_msg(struct idpf_adapter *adapter, uint32_t ops,
uint16_t buf_len, uint8_t *buf);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index ced64dfaf6..5088d50b3e 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -230,6 +230,10 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_DISABLE_QUEUES:
case VIRTCHNL2_OP_ENABLE_VPORT:
case VIRTCHNL2_OP_DISABLE_VPORT:
+ case VIRTCHNL2_OP_MAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR:
+ case VIRTCHNL2_OP_ALLOC_VECTORS:
+ case VIRTCHNL2_OP_DEALLOC_VECTORS:
/* for init virtchnl ops, need to poll the response */
err = idpf_read_one_msg(adapter, args->ops, args->out_size, args->out_buffer);
clear_cmd(adapter);
@@ -521,6 +525,8 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
+
args.ops = VIRTCHNL2_OP_GET_CAPS;
args.in_args = (uint8_t *)&caps_msg;
args.in_args_size = sizeof(caps_msg);
@@ -970,6 +976,113 @@ idpf_vc_config_txq(struct idpf_vport *vport, uint16_t txq_id)
return err;
}
+int
+idpf_vc_config_irq_map_unmap(struct idpf_vport *vport, bool map)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_queue_vector_maps *map_info;
+ struct virtchnl2_queue_vector *vecmap;
+ uint16_t nb_rxq = vport->dev_data->nb_rx_queues;
+ struct idpf_cmd_info args;
+ int len, i, err = 0;
+
+ len = sizeof(struct virtchnl2_queue_vector_maps) +
+ (nb_rxq - 1) * sizeof(struct virtchnl2_queue_vector);
+
+ map_info = rte_zmalloc("map_info", len, 0);
+ if (map_info == NULL)
+ return -ENOMEM;
+
+ map_info->vport_id = vport->vport_id;
+ map_info->num_qv_maps = nb_rxq;
+ for (i = 0; i < nb_rxq; i++) {
+ vecmap = &map_info->qv_maps[i];
+ vecmap->queue_id = vport->qv_map[i].queue_id;
+ vecmap->vector_id = vport->qv_map[i].vector_id;
+ vecmap->itr_idx = VIRTCHNL2_ITR_IDX_0;
+ vecmap->queue_type = VIRTCHNL2_QUEUE_TYPE_RX;
+ }
+
+ args.ops = map ? VIRTCHNL2_OP_MAP_QUEUE_VECTOR :
+ VIRTCHNL2_OP_UNMAP_QUEUE_VECTOR;
+ args.in_args = (u8 *)map_info;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_%s_QUEUE_VECTOR",
+ map ? "MAP" : "UNMAP");
+
+ rte_free(map_info);
+ return err;
+}
+
+int
+idpf_vc_alloc_vectors(struct idpf_vport *vport, uint16_t num_vectors)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ len = sizeof(struct virtchnl2_alloc_vectors) +
+ (num_vectors - 1) * sizeof(struct virtchnl2_vector_chunk);
+ alloc_vec = rte_zmalloc("alloc_vec", len, 0);
+ if (alloc_vec == NULL)
+ return -ENOMEM;
+
+ alloc_vec->num_vectors = num_vectors;
+
+ args.ops = VIRTCHNL2_OP_ALLOC_VECTORS;
+ args.in_args = (u8 *)alloc_vec;
+ args.in_args_size = sizeof(struct virtchnl2_alloc_vectors);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
+
+ if (vport->recv_vectors == NULL) {
+ vport->recv_vectors = rte_zmalloc("recv_vectors", len, 0);
+ if (vport->recv_vectors == NULL) {
+ rte_free(alloc_vec);
+ return -ENOMEM;
+ }
+ }
+
+ rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ rte_free(alloc_vec);
+ return err;
+}
+
+int
+idpf_vc_dealloc_vectors(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_alloc_vectors *alloc_vec;
+ struct virtchnl2_vector_chunks *vcs;
+ struct idpf_cmd_info args;
+ int err, len;
+
+ alloc_vec = vport->recv_vectors;
+ vcs = &alloc_vec->vchunks;
+
+ len = sizeof(struct virtchnl2_vector_chunks) +
+ (vcs->num_vchunks - 1) * sizeof(struct virtchnl2_vector_chunk);
+
+ args.ops = VIRTCHNL2_OP_DEALLOC_VECTORS;
+ args.in_args = (u8 *)vcs;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_DEALLOC_VECTORS");
+
+ return err;
+}
+
static int
idpf_vc_ena_dis_one_queue(struct idpf_vport *vport, uint16_t qid,
uint32_t type, bool on)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 14/18] net/idpf: add support for RSS
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (12 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 15/18] net/idpf: add support for Rx offloading beilei.xing
` (4 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add RSS support.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
drivers/net/idpf/idpf_ethdev.c | 120 ++++++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_ethdev.h | 26 +++++++
drivers/net/idpf/idpf_vchnl.c | 113 +++++++++++++++++++++++++++++++
3 files changed, 258 insertions(+), 1 deletion(-)
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 957cc10616..58560ea404 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -59,6 +59,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->max_mtu = dev_info->max_rx_pktlen - IDPF_ETH_OVERHEAD;
dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+ dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
@@ -169,6 +171,8 @@ idpf_parse_devarg_id(char *name)
return val;
}
+#define IDPF_RSS_KEY_LEN 52
+
static int
idpf_init_vport(struct rte_eth_dev *dev)
{
@@ -189,6 +193,10 @@ idpf_init_vport(struct rte_eth_dev *dev)
vport->max_mtu = vport_info->max_mtu;
rte_memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
+ vport->rss_algorithm = vport_info->rss_algorithm;
+ vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
+ vport_info->rss_key_size);
+ vport->rss_lut_size = vport_info->rss_lut_size;
vport->sw_idx = idx;
for (i = 0; i < vport_info->chunks.num_chunks; i++) {
@@ -246,17 +254,110 @@ idpf_init_vport(struct rte_eth_dev *dev)
return 0;
}
+static int
+idpf_config_rss(struct idpf_vport *vport)
+{
+ int ret;
+
+ ret = idpf_vc_set_rss_key(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS key");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_lut(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS lut");
+ return ret;
+ }
+
+ ret = idpf_vc_set_rss_hash(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS hash");
+ return ret;
+ }
+
+ return ret;
+}
+
+static int
+idpf_init_rss(struct idpf_vport *vport)
+{
+ struct rte_eth_rss_conf *rss_conf;
+ uint16_t i, nb_q, lut_size;
+ int ret = 0;
+
+ rss_conf = &vport->dev_data->dev_conf.rx_adv_conf.rss_conf;
+ nb_q = vport->dev_data->nb_rx_queues;
+
+ vport->rss_key = rte_zmalloc("rss_key",
+ vport->rss_key_size, 0);
+ if (vport->rss_key == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS key");
+ ret = -ENOMEM;
+ goto err_alloc_key;
+ }
+
+ lut_size = vport->rss_lut_size;
+ vport->rss_lut = rte_zmalloc("rss_lut",
+ sizeof(uint32_t) * lut_size, 0);
+ if (vport->rss_lut == NULL) {
+ PMD_INIT_LOG(ERR, "Failed to allocate RSS lut");
+ ret = -ENOMEM;
+ goto err_alloc_lut;
+ }
+
+ if (rss_conf->rss_key == NULL) {
+ for (i = 0; i < vport->rss_key_size; i++)
+ vport->rss_key[i] = (uint8_t)rte_rand();
+ } else if (rss_conf->rss_key_len != vport->rss_key_size) {
+ PMD_INIT_LOG(ERR, "Invalid RSS key length in RSS configuration, should be %d",
+ vport->rss_key_size);
+ ret = -EINVAL;
+ goto err_cfg_key;
+ } else {
+ rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ vport->rss_key_size);
+ }
+
+ for (i = 0; i < lut_size; i++)
+ vport->rss_lut[i] = i % nb_q;
+
+ vport->rss_hf = IDPF_DEFAULT_RSS_HASH_EXPANDED;
+
+ ret = idpf_config_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to configure RSS");
+ goto err_cfg_key;
+ }
+
+ return ret;
+
+err_cfg_key:
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+err_alloc_lut:
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+err_alloc_key:
+ return ret;
+}
+
static int
idpf_dev_configure(struct rte_eth_dev *dev)
{
+ struct idpf_vport *vport = dev->data->dev_private;
struct rte_eth_conf *conf = &dev->data->dev_conf;
+ struct idpf_adapter *adapter = vport->adapter;
+ int ret;
if (conf->link_speeds & RTE_ETH_LINK_SPEED_FIXED) {
PMD_INIT_LOG(ERR, "Setting link speed is not supported");
return -ENOTSUP;
}
- if (dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) {
+ if ((dev->data->nb_rx_queues == 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_NONE) ||
+ (dev->data->nb_rx_queues > 1 && conf->rxmode.mq_mode != RTE_ETH_MQ_RX_RSS)) {
PMD_INIT_LOG(ERR, "Multi-queue packet distribution mode %d is not supported",
conf->rxmode.mq_mode);
return -ENOTSUP;
@@ -294,6 +395,17 @@ idpf_dev_configure(struct rte_eth_dev *dev)
return -ENOTSUP;
}
+ if (adapter->caps->rss_caps != 0 && dev->data->nb_rx_queues != 0) {
+ ret = idpf_init_rss(vport);
+ if (ret != 0) {
+ PMD_INIT_LOG(ERR, "Failed to init rss");
+ return ret;
+ }
+ } else {
+ PMD_INIT_LOG(ERR, "RSS is not supported.");
+ return -1;
+ }
+
return 0;
}
@@ -500,6 +612,12 @@ idpf_dev_close(struct rte_eth_dev *dev)
idpf_vc_destroy_vport(vport);
+ rte_free(vport->rss_lut);
+ vport->rss_lut = NULL;
+
+ rte_free(vport->rss_key);
+ vport->rss_key = NULL;
+
rte_free(vport->recv_vectors);
vport->recv_vectors = NULL;
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 811240c386..8d0804f603 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -48,6 +48,20 @@
#define IDPF_ETH_OVERHEAD \
(RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + IDPF_VLAN_TAG_SIZE * 2)
+#define IDPF_RSS_OFFLOAD_ALL ( \
+ RTE_ETH_RSS_IPV4 | \
+ RTE_ETH_RSS_FRAG_IPV4 | \
+ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV4_OTHER | \
+ RTE_ETH_RSS_IPV6 | \
+ RTE_ETH_RSS_FRAG_IPV6 | \
+ RTE_ETH_RSS_NONFRAG_IPV6_TCP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_UDP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_SCTP | \
+ RTE_ETH_RSS_NONFRAG_IPV6_OTHER)
+
#define IDPF_ADAPTER_NAME_LEN (PCI_PRI_STR_SIZE + 1)
/* Message type read in virtual channel from PF */
@@ -90,11 +104,20 @@ struct idpf_vport {
uint16_t max_mtu;
uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
+ enum virtchnl_rss_algorithm rss_algorithm;
+ uint16_t rss_key_size;
+ uint16_t rss_lut_size;
+
uint16_t sw_idx; /* SW idx */
struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
uint16_t max_pkt_len; /* Maximum packet length */
+ /* RSS info */
+ uint32_t *rss_lut;
+ uint8_t *rss_key;
+ uint64_t rss_hf;
+
/* MSIX info*/
struct virtchnl2_queue_vector *qv_map; /* queue vector mapping */
uint16_t max_vectors;
@@ -200,6 +223,9 @@ int idpf_get_pkt_type(struct idpf_adapter *adapter);
int idpf_vc_get_caps(struct idpf_adapter *adapter);
int idpf_vc_create_vport(struct idpf_adapter *adapter);
int idpf_vc_destroy_vport(struct idpf_vport *vport);
+int idpf_vc_set_rss_key(struct idpf_vport *vport);
+int idpf_vc_set_rss_lut(struct idpf_vport *vport);
+int idpf_vc_set_rss_hash(struct idpf_vport *vport);
int idpf_vc_config_rxqs(struct idpf_vport *vport);
int idpf_vc_config_rxq(struct idpf_vport *vport, uint16_t rxq_id);
int idpf_vc_config_txqs(struct idpf_vport *vport);
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 5088d50b3e..4ef354df89 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -224,6 +224,9 @@ idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info *args)
case VIRTCHNL2_OP_GET_CAPS:
case VIRTCHNL2_OP_CREATE_VPORT:
case VIRTCHNL2_OP_DESTROY_VPORT:
+ case VIRTCHNL2_OP_SET_RSS_KEY:
+ case VIRTCHNL2_OP_SET_RSS_LUT:
+ case VIRTCHNL2_OP_SET_RSS_HASH:
case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
case VIRTCHNL2_OP_ENABLE_QUEUES:
@@ -525,6 +528,22 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.rss_caps =
+ VIRTCHNL2_CAP_RSS_IPV4_TCP |
+ VIRTCHNL2_CAP_RSS_IPV4_UDP |
+ VIRTCHNL2_CAP_RSS_IPV4_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV4_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV6_TCP |
+ VIRTCHNL2_CAP_RSS_IPV6_UDP |
+ VIRTCHNL2_CAP_RSS_IPV6_SCTP |
+ VIRTCHNL2_CAP_RSS_IPV6_OTHER |
+ VIRTCHNL2_CAP_RSS_IPV4_AH |
+ VIRTCHNL2_CAP_RSS_IPV4_ESP |
+ VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH |
+ VIRTCHNL2_CAP_RSS_IPV6_ESP |
+ VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
+
caps_msg.other_caps = VIRTCHNL2_CAP_WB_ON_ITR;
args.ops = VIRTCHNL2_OP_GET_CAPS;
@@ -615,6 +634,100 @@ idpf_vc_destroy_vport(struct idpf_vport *vport)
return err;
}
+int
+idpf_vc_set_rss_key(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_key *rss_key;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_key) + sizeof(rss_key->key[0]) *
+ (vport->rss_key_size - 1);
+ rss_key = rte_zmalloc("rss_key", len, 0);
+ if (rss_key == NULL)
+ return -ENOMEM;
+
+ rss_key->vport_id = vport->vport_id;
+ rss_key->key_len = vport->rss_key_size;
+ rte_memcpy(rss_key->key, vport->rss_key,
+ sizeof(rss_key->key[0]) * vport->rss_key_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_KEY;
+ args.in_args = (uint8_t *)rss_key;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_KEY");
+
+ rte_free(rss_key);
+ return err;
+}
+
+int
+idpf_vc_set_rss_lut(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_lut *rss_lut;
+ struct idpf_cmd_info args;
+ int len, err;
+
+ len = sizeof(*rss_lut) + sizeof(rss_lut->lut[0]) *
+ (vport->rss_lut_size - 1);
+ rss_lut = rte_zmalloc("rss_lut", len, 0);
+ if (rss_lut == NULL)
+ return -ENOMEM;
+
+ rss_lut->vport_id = vport->vport_id;
+ rss_lut->lut_entries = vport->rss_lut_size;
+ rte_memcpy(rss_lut->lut, vport->rss_lut,
+ sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_LUT;
+ args.in_args = (uint8_t *)rss_lut;
+ args.in_args_size = len;
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_SET_RSS_LUT");
+
+ rte_free(rss_lut);
+ return err;
+}
+
+int
+idpf_vc_set_rss_hash(struct idpf_vport *vport)
+{
+ struct idpf_adapter *adapter = vport->adapter;
+ struct virtchnl2_rss_hash rss_hash;
+ struct idpf_cmd_info args;
+ int err;
+
+ memset(&rss_hash, 0, sizeof(rss_hash));
+ rss_hash.ptype_groups = vport->rss_hf;
+ rss_hash.vport_id = vport->vport_id;
+
+ memset(&args, 0, sizeof(args));
+ args.ops = VIRTCHNL2_OP_SET_RSS_HASH;
+ args.in_args = (uint8_t *)&rss_hash;
+ args.in_args_size = sizeof(rss_hash);
+ args.out_buffer = adapter->mbx_resp;
+ args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
+
+ err = idpf_execute_vc_cmd(adapter, &args);
+ if (err != 0)
+ PMD_DRV_LOG(ERR, "Failed to execute command of OP_SET_RSS_HASH");
+
+ return err;
+}
+
#define IDPF_RX_BUF_STRIDE 64
int
idpf_vc_config_rxqs(struct idpf_vport *vport)
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 15/18] net/idpf: add support for Rx offloading
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (13 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 14/18] net/idpf: add support for RSS beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 16/18] net/idpf: add support for Tx offloading beilei.xing
` (3 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Rx offloading support:
- support CHKSUM and RSS offload for split queue model
- support CHKSUM offload for single queue model
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 5 ++
drivers/net/idpf/idpf_ethdev.c | 6 ++
drivers/net/idpf/idpf_rxtx.c | 123 ++++++++++++++++++++++++++++++
drivers/net/idpf/idpf_vchnl.c | 18 +++++
4 files changed, 152 insertions(+)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d722c49fde..868571654f 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -3,8 +3,13 @@
;
; Refer to default.ini for the full list of available PMD features.
;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
[Features]
MTU update = Y
+L3 checksum offload = P
+L4 checksum offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 58560ea404..a09f104425 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -61,6 +61,12 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->flow_type_rss_offloads = IDPF_RSS_OFFLOAD_ALL;
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+
dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index a980714060..f15e61a785 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1209,6 +1209,73 @@ idpf_stop_queues(struct rte_eth_dev *dev)
}
}
+#define IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S))
+
+static inline uint64_t
+idpf_splitq_rx_csum_offload(uint8_t err)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((err & IDPF_RX_FLEX_DESC_ADV_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((err & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
+#define IDPF_RX_FLEX_DESC_ADV_HASH1_S 0
+#define IDPF_RX_FLEX_DESC_ADV_HASH2_S 16
+#define IDPF_RX_FLEX_DESC_ADV_HASH3_S 24
+
+static inline uint64_t
+idpf_splitq_rx_rss_offload(struct rte_mbuf *mb,
+ volatile struct virtchnl2_rx_flex_desc_adv_nic_3 *rx_desc)
+{
+ uint8_t status_err0_qw0;
+ uint64_t flags = 0;
+
+ status_err0_qw0 = rx_desc->status_err0_qw0;
+
+ if ((status_err0_qw0 & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_ADV_STATUS0_RSS_VALID_S)) != 0) {
+ flags |= RTE_MBUF_F_RX_RSS_HASH;
+ mb->hash.rss = (rte_le_to_cpu_16(rx_desc->hash1) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH1_S) |
+ ((uint32_t)(rx_desc->ff2_mirrid_hash2.hash2) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH2_S) |
+ ((uint32_t)(rx_desc->hash3) <<
+ IDPF_RX_FLEX_DESC_ADV_HASH3_S);
+ }
+
+ return flags;
+}
+
static void
idpf_split_rx_bufq_refill(struct idpf_rx_queue *rx_bufq)
{
@@ -1282,9 +1349,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t pktlen_gen_bufq_id;
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
+ uint8_t status_err0_qw1;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
+ uint64_t pkt_flags;
uint16_t pkt_len;
uint16_t bufq_id;
uint16_t gen_id;
@@ -1349,11 +1418,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->next = NULL;
rxm->nb_segs = 1;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
rxm->packet_type =
ptype_tbl[(rte_le_to_cpu_16(rx_desc->ptype_err_fflags0) &
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_M) >>
VIRTCHNL2_RX_FLEX_DESC_ADV_PTYPE_S];
+ status_err0_qw1 = rx_desc->status_err0_qw1;
+ pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
+ pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
@@ -1513,6 +1589,48 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return nb_tx;
}
+#define IDPF_RX_FLEX_DESC_STATUS0_XSUM_S \
+ (RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S) | \
+ RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S))
+
+/* Translate the rx descriptor status and error fields to pkt flags */
+static inline uint64_t
+idpf_rxd_to_pkt_flags(uint16_t status_error)
+{
+ uint64_t flags = 0;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_L3L4P_S)) == 0))
+ return flags;
+
+ if (likely((status_error & IDPF_RX_FLEX_DESC_STATUS0_XSUM_S) == 0)) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD);
+ return flags;
+ }
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_IPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_L4E_S)) != 0))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+ if (unlikely((status_error & RTE_BIT32(VIRTCHNL2_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S)) != 0))
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+ else
+ flags |= RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+
+ return flags;
+}
+
static inline void
idpf_update_rx_tail(struct idpf_rx_queue *rxq, uint16_t nb_hold,
uint16_t rx_id)
@@ -1546,6 +1664,7 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
+ uint64_t pkt_flags;
uint64_t dma_addr;
uint16_t nb_rx;
@@ -1611,10 +1730,14 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->pkt_len = rx_packet_len;
rxm->data_len = rx_packet_len;
rxm->port = rxq->port_id;
+ rxm->ol_flags = 0;
+ pkt_flags = idpf_rxd_to_pkt_flags(rx_status0);
rxm->packet_type =
ptype_tbl[(uint8_t)(rte_cpu_to_le_16(rxd.flex_nic_wb.ptype_flex_flags0) &
VIRTCHNL2_RX_FLEX_DESC_PTYPE_M)];
+ rxm->ol_flags |= pkt_flags;
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c
index 4ef354df89..00ac5b2a6b 100644
--- a/drivers/net/idpf/idpf_vchnl.c
+++ b/drivers/net/idpf/idpf_vchnl.c
@@ -528,6 +528,24 @@ idpf_vc_get_caps(struct idpf_adapter *adapter)
memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
+ caps_msg.csum_caps =
+ VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_TX_CSUM_GENERIC |
+ VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
+ VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
+ VIRTCHNL2_CAP_RX_CSUM_GENERIC;
+
caps_msg.rss_caps =
VIRTCHNL2_CAP_RSS_IPV4_TCP |
VIRTCHNL2_CAP_RSS_IPV4_UDP |
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 16/18] net/idpf: add support for Tx offloading
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (14 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 15/18] net/idpf: add support for Rx offloading beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
` (2 subsequent siblings)
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Xiaoyun Li
From: Junfeng Guo <junfeng.guo@intel.com>
Add Tx offloading support:
- support TSO for single queue model and split queue model.
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 4 +-
drivers/net/idpf/idpf_rxtx.c | 128 +++++++++++++++++++++++++++++-
drivers/net/idpf/idpf_rxtx.h | 22 +++++
4 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index 868571654f..d82b4aa0ff 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -8,6 +8,7 @@
;
[Features]
MTU update = Y
+TSO = P
L3 checksum offload = P
L4 checksum offload = P
Linux = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index a09f104425..084426260c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -67,7 +67,9 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
- dev_info->tx_offload_capa = RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index f15e61a785..cc296d7ab1 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -1506,6 +1506,49 @@ idpf_split_tx_free(struct idpf_tx_queue *cq)
cq->tx_tail = next;
}
+/* Check if the context descriptor is needed for TX offloading */
+static inline uint16_t
+idpf_calc_context_desc(uint64_t flags)
+{
+ if ((flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ return 1;
+
+ return 0;
+}
+
+/* set TSO context descriptor
+ */
+static inline void
+idpf_set_splitq_tso_ctx(struct rte_mbuf *mbuf,
+ union idpf_tx_offload tx_offload,
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc)
+{
+ uint16_t cmd_dtype;
+ uint32_t tso_len;
+ uint8_t hdr_len;
+
+ if (tx_offload.l4_len == 0) {
+ PMD_TX_LOG(DEBUG, "L4 length set to 0");
+ return;
+ }
+
+ hdr_len = tx_offload.l2_len +
+ tx_offload.l3_len +
+ tx_offload.l4_len;
+ cmd_dtype = IDPF_TX_DESC_DTYPE_FLEX_TSO_CTX |
+ IDPF_TX_FLEX_CTX_DESC_CMD_TSO;
+ tso_len = mbuf->pkt_len - hdr_len;
+
+ ctx_desc->tso.qw1.cmd_dtype = rte_cpu_to_le_16(cmd_dtype);
+ ctx_desc->tso.qw0.hdr_len = hdr_len;
+ ctx_desc->tso.qw0.mss_rt =
+ rte_cpu_to_le_16((uint16_t)mbuf->tso_segsz &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+ ctx_desc->tso.qw0.flex_tlen =
+ rte_cpu_to_le_32(tso_len &
+ IDPF_TXD_FLEX_CTX_MSS_RT_M);
+}
+
uint16_t
idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
@@ -1514,11 +1557,14 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
volatile struct idpf_flex_tx_sched_desc *txr;
volatile struct idpf_flex_tx_sched_desc *txd;
struct idpf_tx_entry *sw_ring;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
uint16_t nb_used, tx_id, sw_id;
struct rte_mbuf *tx_pkt;
uint16_t nb_to_clean;
uint16_t nb_tx = 0;
+ uint64_t ol_flags;
+ uint16_t nb_ctx;
if (unlikely(txq == NULL) || unlikely(!txq->q_started))
return nb_tx;
@@ -1548,7 +1594,29 @@ idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
if (txq->nb_free < tx_pkt->nb_segs)
break;
- nb_used = tx_pkt->nb_segs;
+
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+ nb_used = tx_pkt->nb_segs + nb_ctx;
+
+ /* context descriptor */
+ if (nb_ctx != 0) {
+ volatile union idpf_flex_tx_ctx_desc *ctx_desc =
+ (volatile union idpf_flex_tx_ctx_desc *)&txr[tx_id];
+
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_desc);
+
+ tx_id++;
+ if (tx_id == txq->nb_tx_desc)
+ tx_id = 0;
+ }
do {
txd = &txr[tx_id];
@@ -1799,14 +1867,17 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
{
volatile struct idpf_flex_tx_desc *txd;
volatile struct idpf_flex_tx_desc *txr;
+ union idpf_tx_offload tx_offload = {0};
struct idpf_tx_entry *txe, *txn;
struct idpf_tx_entry *sw_ring;
struct idpf_tx_queue *txq;
struct rte_mbuf *tx_pkt;
struct rte_mbuf *m_seg;
uint64_t buf_dma_addr;
+ uint64_t ol_flags;
uint16_t tx_last;
uint16_t nb_used;
+ uint16_t nb_ctx;
uint16_t td_cmd;
uint16_t tx_id;
uint16_t nb_tx;
@@ -1833,11 +1904,19 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
tx_pkt = *tx_pkts++;
RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf);
+ ol_flags = tx_pkt->ol_flags;
+ tx_offload.l2_len = tx_pkt->l2_len;
+ tx_offload.l3_len = tx_pkt->l3_len;
+ tx_offload.l4_len = tx_pkt->l4_len;
+ tx_offload.tso_segsz = tx_pkt->tso_segsz;
+ /* Calculate the number of context descriptors needed. */
+ nb_ctx = idpf_calc_context_desc(ol_flags);
+
/* The number of descriptors that must be allocated for
* a packet equals to the number of the segments of that
* packet plus 1 context descriptor if needed.
*/
- nb_used = (uint16_t)(tx_pkt->nb_segs);
+ nb_used = (uint16_t)(tx_pkt->nb_segs + nb_ctx);
tx_last = (uint16_t)(tx_id + nb_used - 1);
/* Circular ring */
@@ -1865,6 +1944,29 @@ idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
}
}
+ if (nb_ctx != 0) {
+ /* Setup TX context descriptor if required */
+ volatile union idpf_flex_tx_ctx_desc *ctx_txd =
+ (volatile union idpf_flex_tx_ctx_desc *)
+ &txr[tx_id];
+
+ txn = &sw_ring[txe->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(txn->mbuf);
+ if (txe->mbuf != NULL) {
+ rte_pktmbuf_free_seg(txe->mbuf);
+ txe->mbuf = NULL;
+ }
+
+ /* TSO enabled */
+ if ((ol_flags & RTE_MBUF_F_TX_TCP_SEG) != 0)
+ idpf_set_splitq_tso_ctx(tx_pkt, tx_offload,
+ ctx_txd);
+
+ txe->last_id = tx_last;
+ tx_id = txe->next_id;
+ txe = txn;
+ }
+
m_seg = tx_pkt;
do {
txd = &txr[tx_id];
@@ -1924,6 +2026,9 @@ uint16_t
idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts)
{
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ int ret;
+#endif
int i;
uint64_t ol_flags;
struct rte_mbuf *m;
@@ -1938,12 +2043,31 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
rte_errno = EINVAL;
return i;
}
+ } else if ((m->tso_segsz < IDPF_MIN_TSO_MSS) ||
+ (m->tso_segsz > IDPF_MAX_TSO_MSS) ||
+ (m->pkt_len > IDPF_MAX_TSO_FRAME_SIZE)) {
+ /* MSS outside the range are considered malicious */
+ rte_errno = EINVAL;
+ return i;
+ }
+
+ if ((ol_flags & IDPF_TX_OFFLOAD_NOTSUP_MASK) != 0) {
+ rte_errno = ENOTSUP;
+ return i;
}
if (m->pkt_len < IDPF_MIN_FRAME_SIZE) {
rte_errno = EINVAL;
return i;
}
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = -ret;
+ return i;
+ }
+#endif
}
return i;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 3853ed55c9..54d297aac6 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -23,6 +23,16 @@
#define IDPF_TX_MAX_MTU_SEG 10
+#define IDPF_MIN_TSO_MSS 88
+#define IDPF_MAX_TSO_MSS 9728
+#define IDPF_MAX_TSO_FRAME_SIZE 262143
+#define IDPF_TX_MAX_MTU_SEG 10
+
+#define IDPF_TX_OFFLOAD_MASK RTE_MBUF_F_TX_TCP_SEG
+
+#define IDPF_TX_OFFLOAD_NOTSUP_MASK \
+ (RTE_MBUF_F_TX_OFFLOAD_MASK ^ IDPF_TX_OFFLOAD_MASK)
+
#define IDPF_GET_PTYPE_SIZE(p) \
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
@@ -115,6 +125,18 @@ struct idpf_tx_queue {
struct idpf_tx_queue *complq;
};
+/* Offload features */
+union idpf_tx_offload {
+ uint64_t data;
+ struct {
+ uint64_t l2_len:7; /* L2 (MAC) Header Length. */
+ uint64_t l3_len:9; /* L3 (IP) Header Length. */
+ uint64_t l4_len:8; /* L4 Header Length. */
+ uint64_t tso_segsz:16; /* TCP TSO segment size */
+ /* uint64_t unused : 24; */
+ };
+};
+
struct idpf_rxq_ops {
void (*release_mbufs)(struct idpf_rx_queue *rxq);
};
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 17/18] net/idpf: add AVX512 data path for single queue model
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (15 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 16/18] net/idpf: add support for Tx offloading beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 8:33 ` [PATCH v18 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 13:38 ` [PATCH v18 00/18] add support for idpf PMD in DPDK Thomas Monjalon
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjun Wu
From: Junfeng Guo <junfeng.guo@intel.com>
Add support of AVX512 vector data path for single queue model.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/idpf.rst | 19 +
drivers/net/idpf/idpf_ethdev.c | 3 +-
drivers/net/idpf/idpf_ethdev.h | 5 +
drivers/net/idpf/idpf_rxtx.c | 145 ++++
drivers/net/idpf/idpf_rxtx.h | 21 +
drivers/net/idpf/idpf_rxtx_vec_avx512.c | 857 ++++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx_vec_common.h | 100 +++
drivers/net/idpf/meson.build | 28 +
8 files changed, 1177 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_avx512.c
create mode 100644 drivers/net/idpf/idpf_rxtx_vec_common.h
diff --git a/doc/guides/nics/idpf.rst b/doc/guides/nics/idpf.rst
index c1001d5d0c..3039c61748 100644
--- a/doc/guides/nics/idpf.rst
+++ b/doc/guides/nics/idpf.rst
@@ -64,3 +64,22 @@ Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and
for details.
+Features
+--------
+
+Vector PMD
+~~~~~~~~~~
+
+Vector path for RX and TX path are selected automatically. The paths
+are chosen based on 2 conditions.
+
+- ``CPU``
+ On the X86 platform, the driver checks if the CPU supports AVX512.
+ If the CPU supports AVX512 and EAL argument ``--force-max-simd-bitwidth``
+ is set to 512, AVX512 paths will be chosen.
+
+- ``Offload features``
+ The supported HW offload features are described in the document idpf.ini,
+ A value "P" means the offload feature is not supported by vector path.
+ If any not supported features are used, idpf vector PMD is disabled and the
+ scalar paths are chosen.
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 084426260c..cd4ebcc2c6 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -69,7 +69,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
- RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
dev_info->default_txconf = (struct rte_eth_txconf) {
.tx_free_thresh = IDPF_DEFAULT_TX_FREE_THRESH,
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 8d0804f603..7d54e5db60 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -162,6 +162,11 @@ struct idpf_adapter {
uint32_t max_txq_per_msg;
uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE] __rte_cache_min_aligned;
+
+ bool rx_vec_allowed;
+ bool tx_vec_allowed;
+ bool rx_use_avx512;
+ bool tx_use_avx512;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index cc296d7ab1..9e20f2b9d3 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -4,9 +4,11 @@
#include <ethdev_driver.h>
#include <rte_net.h>
+#include <rte_vect.h>
#include "idpf_ethdev.h"
#include "idpf_rxtx.h"
+#include "idpf_rxtx_vec_common.h"
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
@@ -252,6 +254,8 @@ reset_single_rx_queue(struct idpf_rx_queue *rxq)
rxq->pkt_first_seg = NULL;
rxq->pkt_last_seg = NULL;
+ rxq->rxrearm_start = 0;
+ rxq->rxrearm_nb = 0;
}
static void
@@ -2073,25 +2077,166 @@ idpf_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
return i;
}
+static void __rte_cold
+release_rxq_mbufs_vec(struct idpf_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->nb_rx_desc - 1;
+ uint16_t i;
+
+ if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
+ return;
+
+ /* free all mbufs that are valid in the ring */
+ if (rxq->rxrearm_nb == 0) {
+ for (i = 0; i < rxq->nb_rx_desc; i++) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ } else {
+ for (i = rxq->rx_tail; i != rxq->rxrearm_start; i = (i + 1) & mask) {
+ if (rxq->sw_ring[i] != NULL)
+ rte_pktmbuf_free_seg(rxq->sw_ring[i]);
+ }
+ }
+
+ rxq->rxrearm_nb = rxq->nb_rx_desc;
+
+ /* set all entries to NULL */
+ memset(rxq->sw_ring, 0, sizeof(rxq->sw_ring[0]) * rxq->nb_rx_desc);
+}
+
+static const struct idpf_rxq_ops def_singleq_rx_ops_vec = {
+ .release_mbufs = release_rxq_mbufs_vec,
+};
+
+static inline int
+idpf_singleq_rx_vec_setup_default(struct idpf_rx_queue *rxq)
+{
+ uintptr_t p;
+ struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
+
+ mb_def.nb_segs = 1;
+ mb_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mb_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mb_def, 1);
+
+ /* prevent compiler reordering: rearm_data covers previous fields */
+ rte_compiler_barrier();
+ p = (uintptr_t)&mb_def.rearm_data;
+ rxq->mbuf_initializer = *(uint64_t *)p;
+ return 0;
+}
+
+int __rte_cold
+idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq)
+{
+ rxq->ops = &def_singleq_rx_ops_vec;
+ return idpf_singleq_rx_vec_setup_default(rxq);
+}
+
void
idpf_set_rx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+ struct idpf_rx_queue *rxq;
+ int i;
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->rx_vec_allowed = true;
+
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->rx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->rx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
+#ifdef RTE_ARCH_X86
+ if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
+ dev->rx_pkt_burst = idpf_splitq_recv_pkts;
+ } else {
+ if (ad->rx_vec_allowed) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ (void)idpf_singleq_rx_vec_setup(rxq);
+ }
+#ifdef CC_AVX512_SUPPORT
+ if (ad->rx_use_avx512) {
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts_avx512;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+ dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+ }
+#else
if (vport->rxq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT)
dev->rx_pkt_burst = idpf_splitq_recv_pkts;
else
dev->rx_pkt_burst = idpf_singleq_recv_pkts;
+#endif /* RTE_ARCH_X86 */
}
void
idpf_set_tx_function(struct rte_eth_dev *dev)
{
struct idpf_vport *vport = dev->data->dev_private;
+#ifdef RTE_ARCH_X86
+ struct idpf_adapter *ad = vport->adapter;
+#ifdef CC_AVX512_SUPPORT
+ struct idpf_tx_queue *txq;
+ int i;
+#endif /* CC_AVX512_SUPPORT */
+
+ if (idpf_rx_vec_dev_check_default(dev) == IDPF_VECTOR_PATH &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ ad->tx_vec_allowed = true;
+ if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+#ifdef CC_AVX512_SUPPORT
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1 &&
+ rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)
+ ad->tx_use_avx512 = true;
+#else
+ PMD_DRV_LOG(NOTICE,
+ "AVX512 is not supported in build env");
+#endif /* CC_AVX512_SUPPORT */
+ } else {
+ ad->tx_vec_allowed = false;
+ }
+#endif /* RTE_ARCH_X86 */
+
if (vport->txq_model == VIRTCHNL2_QUEUE_MODEL_SPLIT) {
dev->tx_pkt_burst = idpf_splitq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
} else {
+#ifdef RTE_ARCH_X86
+ if (ad->tx_vec_allowed) {
+#ifdef CC_AVX512_SUPPORT
+ if (ad->tx_use_avx512) {
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL)
+ continue;
+ idpf_singleq_tx_vec_setup_avx512(txq);
+ }
+ dev->tx_pkt_burst = idpf_singleq_xmit_pkts_avx512;
+ dev->tx_pkt_prepare = idpf_prep_pkts;
+ return;
+ }
+#endif /* CC_AVX512_SUPPORT */
+ }
+#endif /* RTE_ARCH_X86 */
dev->tx_pkt_burst = idpf_singleq_xmit_pkts;
dev->tx_pkt_prepare = idpf_prep_pkts;
}
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 54d297aac6..506bf8a57d 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -18,6 +18,12 @@
#define IDPF_RX_MAX_BURST 32
#define IDPF_DEFAULT_RX_FREE_THRESH 32
+/* used for Vector PMD */
+#define IDPF_VPMD_RX_MAX_BURST 32
+#define IDPF_VPMD_TX_MAX_BURST 32
+#define IDPF_VPMD_DESCS_PER_LOOP 4
+#define IDPF_RXQ_REARM_THRESH 64
+
#define IDPF_DEFAULT_TX_RS_THRESH 32
#define IDPF_DEFAULT_TX_FREE_THRESH 32
@@ -54,6 +60,11 @@ struct idpf_rx_queue {
struct rte_mbuf *pkt_last_seg; /* last segment of current packet */
struct rte_mbuf fake_mbuf; /* dummy mbuf */
+ /* used for VPMD */
+ uint16_t rxrearm_nb; /* number of remaining to be re-armed */
+ uint16_t rxrearm_start; /* the idx we start the re-arming from */
+ uint64_t mbuf_initializer; /* value to init mbufs */
+
uint16_t rx_nb_avail;
uint16_t rx_next_avail;
@@ -84,6 +95,10 @@ struct idpf_tx_entry {
uint16_t last_id;
};
+struct idpf_tx_vec_entry {
+ struct rte_mbuf *mbuf;
+};
+
/* Structure associated with each TX queue. */
struct idpf_tx_queue {
const struct rte_memzone *mz; /* memzone for Tx ring */
@@ -149,6 +164,7 @@ int idpf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *rx_conf,
struct rte_mempool *mp);
+int idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq);
int idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
int idpf_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
@@ -157,16 +173,21 @@ void idpf_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
int idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *tx_conf);
+int idpf_singleq_rx_vec_setup(struct idpf_rx_queue *rxq);
int idpf_tx_queue_init(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id);
int idpf_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
void idpf_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
uint16_t idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
uint16_t idpf_singleq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
uint16_t idpf_splitq_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
uint16_t idpf_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
diff --git a/drivers/net/idpf/idpf_rxtx_vec_avx512.c b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
new file mode 100644
index 0000000000..fb2b6bb53c
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_avx512.c
@@ -0,0 +1,857 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#include "idpf_rxtx_vec_common.h"
+
+#include <rte_vect.h>
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_DESCS_PER_LOOP_AVX 8
+#define PKTLEN_SHIFT 10
+
+static __rte_always_inline void
+idpf_singleq_rearm_common(struct idpf_rx_queue *rxq)
+{
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ uint16_t rx_id;
+ int i;
+
+ rxdp += rxq->rxrearm_start;
+
+ /* Pull 'n' more MBUFs into the software ring */
+ if (rte_mempool_get_bulk(rxq->mp,
+ (void *)rxp,
+ IDPF_RXQ_REARM_THRESH) < 0) {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_store_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ struct rte_mbuf *mb0, *mb1, *mb2, *mb3;
+ struct rte_mbuf *mb4, *mb5, *mb6, *mb7;
+ __m512i dma_addr0_3, dma_addr4_7;
+ __m512i hdr_room = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH;
+ i += 8, rxp += 8, rxdp += 8) {
+ __m128i vaddr0, vaddr1, vaddr2, vaddr3;
+ __m128i vaddr4, vaddr5, vaddr6, vaddr7;
+ __m256i vaddr0_1, vaddr2_3;
+ __m256i vaddr4_5, vaddr6_7;
+ __m512i vaddr0_3, vaddr4_7;
+
+ mb0 = rxp[0];
+ mb1 = rxp[1];
+ mb2 = rxp[2];
+ mb3 = rxp[3];
+ mb4 = rxp[4];
+ mb5 = rxp[5];
+ mb6 = rxp[6];
+ mb7 = rxp[7];
+
+ /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+ vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr);
+ vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr);
+ vaddr2 = _mm_loadu_si128((__m128i *)&mb2->buf_addr);
+ vaddr3 = _mm_loadu_si128((__m128i *)&mb3->buf_addr);
+ vaddr4 = _mm_loadu_si128((__m128i *)&mb4->buf_addr);
+ vaddr5 = _mm_loadu_si128((__m128i *)&mb5->buf_addr);
+ vaddr6 = _mm_loadu_si128((__m128i *)&mb6->buf_addr);
+ vaddr7 = _mm_loadu_si128((__m128i *)&mb7->buf_addr);
+
+ /**
+ * merge 0 & 1, by casting 0 to 256-bit and inserting 1
+ * into the high lanes. Similarly for 2 & 3, and so on.
+ */
+ vaddr0_1 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr0),
+ vaddr1, 1);
+ vaddr2_3 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr2),
+ vaddr3, 1);
+ vaddr4_5 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr4),
+ vaddr5, 1);
+ vaddr6_7 =
+ _mm256_inserti128_si256(_mm256_castsi128_si256(vaddr6),
+ vaddr7, 1);
+ vaddr0_3 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr0_1),
+ vaddr2_3, 1);
+ vaddr4_7 =
+ _mm512_inserti64x4(_mm512_castsi256_si512(vaddr4_5),
+ vaddr6_7, 1);
+
+ /* convert pa to dma_addr hdr/data */
+ dma_addr0_3 = _mm512_unpackhi_epi64(vaddr0_3, vaddr0_3);
+ dma_addr4_7 = _mm512_unpackhi_epi64(vaddr4_7, vaddr4_7);
+
+ /* add headroom to pa values */
+ dma_addr0_3 = _mm512_add_epi64(dma_addr0_3, hdr_room);
+ dma_addr4_7 = _mm512_add_epi64(dma_addr4_7, hdr_room);
+
+ /* flush desc with pa dma_addr */
+ _mm512_store_si512((__m512i *)&rxdp->read, dma_addr0_3);
+ _mm512_store_si512((__m512i *)&(rxdp + 4)->read, dma_addr4_7);
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+static __rte_always_inline void
+idpf_singleq_rearm(struct idpf_rx_queue *rxq)
+{
+ int i;
+ uint16_t rx_id;
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+ struct rte_mempool_cache *cache =
+ rte_mempool_default_cache(rxq->mp, rte_lcore_id());
+ struct rte_mbuf **rxp = &rxq->sw_ring[rxq->rxrearm_start];
+
+ rxdp += rxq->rxrearm_start;
+
+ if (unlikely(cache == NULL))
+ return idpf_singleq_rearm_common(rxq);
+
+ /* We need to pull 'n' more MBUFs into the software ring from mempool
+ * We inline the mempool function here, so we can vectorize the copy
+ * from the cache into the shadow ring.
+ */
+
+ /* Can this be satisfied from the cache? */
+ if (cache->len < IDPF_RXQ_REARM_THRESH) {
+ /* No. Backfill the cache first, and then fill from it */
+ uint32_t req = IDPF_RXQ_REARM_THRESH + (cache->size -
+ cache->len);
+
+ /* How many do we require i.e. number to fill the cache + the request */
+ int ret = rte_mempool_ops_dequeue_bulk
+ (rxq->mp, &cache->objs[cache->len], req);
+ if (ret == 0) {
+ cache->len += req;
+ } else {
+ if (rxq->rxrearm_nb + IDPF_RXQ_REARM_THRESH >=
+ rxq->nb_rx_desc) {
+ __m128i dma_addr0;
+
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < IDPF_VPMD_DESCS_PER_LOOP; i++) {
+ rxp[i] = &rxq->fake_mbuf;
+ _mm_storeu_si128((__m128i *)&rxdp[i].read,
+ dma_addr0);
+ }
+ }
+ rte_eth_devices[rxq->port_id].data->rx_mbuf_alloc_failed +=
+ IDPF_RXQ_REARM_THRESH;
+ return;
+ }
+ }
+
+ const __m512i iova_offsets = _mm512_set1_epi64(offsetof
+ (struct rte_mbuf, buf_iova));
+ const __m512i headroom = _mm512_set1_epi64(RTE_PKTMBUF_HEADROOM);
+
+ /* to shuffle the addresses to correct slots. Values 4-7 will contain
+ * zeros, so use 7 for a zero-value.
+ */
+ const __m512i permute_idx = _mm512_set_epi64(7, 7, 3, 1, 7, 7, 2, 0);
+
+ /* Initialize the mbufs in vector, process 8 mbufs in one loop, taking
+ * from mempool cache and populating both shadow and HW rings
+ */
+ for (i = 0; i < IDPF_RXQ_REARM_THRESH / IDPF_DESCS_PER_LOOP_AVX; i++) {
+ const __m512i mbuf_ptrs = _mm512_loadu_si512
+ (&cache->objs[cache->len - IDPF_DESCS_PER_LOOP_AVX]);
+ _mm512_storeu_si512(rxp, mbuf_ptrs);
+
+ const __m512i iova_base_addrs = _mm512_i64gather_epi64
+ (_mm512_add_epi64(mbuf_ptrs, iova_offsets),
+ 0, /* base */
+ 1 /* scale */);
+ const __m512i iova_addrs = _mm512_add_epi64(iova_base_addrs,
+ headroom);
+ const __m512i iovas0 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 0));
+ const __m512i iovas1 = _mm512_castsi256_si512
+ (_mm512_extracti64x4_epi64(iova_addrs, 1));
+
+ /* permute leaves desc 2-3 addresses in header address slots 0-1
+ * but these are ignored by driver since header split not
+ * enabled. Similarly for desc 6 & 7.
+ */
+ const __m512i desc0_1 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas0);
+ const __m512i desc2_3 = _mm512_bsrli_epi128(desc0_1, 8);
+
+ const __m512i desc4_5 = _mm512_permutexvar_epi64
+ (permute_idx,
+ iovas1);
+ const __m512i desc6_7 = _mm512_bsrli_epi128(desc4_5, 8);
+
+ _mm512_storeu_si512((void *)rxdp, desc0_1);
+ _mm512_storeu_si512((void *)(rxdp + 2), desc2_3);
+ _mm512_storeu_si512((void *)(rxdp + 4), desc4_5);
+ _mm512_storeu_si512((void *)(rxdp + 6), desc6_7);
+
+ rxp += IDPF_DESCS_PER_LOOP_AVX;
+ rxdp += IDPF_DESCS_PER_LOOP_AVX;
+ cache->len -= IDPF_DESCS_PER_LOOP_AVX;
+ }
+
+ rxq->rxrearm_start += IDPF_RXQ_REARM_THRESH;
+ if (rxq->rxrearm_start >= rxq->nb_rx_desc)
+ rxq->rxrearm_start = 0;
+
+ rxq->rxrearm_nb -= IDPF_RXQ_REARM_THRESH;
+
+ rx_id = (uint16_t)((rxq->rxrearm_start == 0) ?
+ (rxq->nb_rx_desc - 1) : (rxq->rxrearm_start - 1));
+
+ /* Update the tail pointer on the NIC */
+ IDPF_PCI_REG_WRITE(rxq->qrx_tail, rx_id);
+}
+
+#define IDPF_RX_LEN_MASK 0x80808080
+static __rte_always_inline uint16_t
+_idpf_singleq_recv_raw_pkts_avx512(struct idpf_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ const uint32_t *type_table = rxq->adapter->ptype_tbl;
+
+ const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0,
+ rxq->mbuf_initializer);
+ struct rte_mbuf **sw_ring = &rxq->sw_ring[rxq->rx_tail];
+ volatile union virtchnl2_rx_desc *rxdp = rxq->rx_ring;
+
+ rxdp += rxq->rx_tail;
+
+ rte_prefetch0(rxdp);
+
+ /* nb_pkts has to be floor-aligned to IDPF_DESCS_PER_LOOP_AVX */
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, IDPF_DESCS_PER_LOOP_AVX);
+
+ /* See if we need to rearm the RX queue - gives the prefetch a bit
+ * of time to act
+ */
+ if (rxq->rxrearm_nb > IDPF_RXQ_REARM_THRESH)
+ idpf_singleq_rearm(rxq);
+
+ /* Before we start moving massive data around, check to see if
+ * there is actually a packet available
+ */
+ if ((rxdp->flex_nic_wb.status_error0 &
+ rte_cpu_to_le_32(1 << VIRTCHNL2_RX_FLEX_DESC_STATUS0_DD_S)) == 0)
+ return 0;
+
+ /* 8 packets DD mask, LSB in each 32-bit value */
+ const __m256i dd_check = _mm256_set1_epi32(1);
+
+ /* mask to shuffle from desc. to mbuf (4 descriptors)*/
+ const __m512i shuf_msk =
+ _mm512_set_epi32
+ (/* 1st descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 2nd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 3rd descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF, /* pkt_type set as unknown */
+ /* 4th descriptor */
+ 0xFFFFFFFF, /* rss set as unknown */
+ 0xFFFF0504, /* vlan_macip set as unknown */
+ /* octet 15~14, 16 bits data_len */
+ 0xFFFF0504, /* skip high 16 bits pkt_len, zero out */
+ /* octet 15~14, low 16 bits pkt_len */
+ 0xFFFFFFFF /* pkt_type set as unknown */
+ );
+ /**
+ * compile-time check the shuffle layout is correct.
+ * NOTE: the first field (lowest address) is given last in set_epi
+ * calls above.
+ */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+
+ uint16_t i, received;
+
+ for (i = 0, received = 0; i < nb_pkts;
+ i += IDPF_DESCS_PER_LOOP_AVX,
+ rxdp += IDPF_DESCS_PER_LOOP_AVX) {
+ /* step 1, copy over 8 mbuf pointers to rx_pkts array */
+ _mm256_storeu_si256((void *)&rx_pkts[i],
+ _mm256_loadu_si256((void *)&sw_ring[i]));
+#ifdef RTE_ARCH_X86_64
+ _mm256_storeu_si256
+ ((void *)&rx_pkts[i + 4],
+ _mm256_loadu_si256((void *)&sw_ring[i + 4]));
+#endif
+
+ __m512i raw_desc0_3, raw_desc4_7;
+ const __m128i raw_desc7 =
+ _mm_load_si128((void *)(rxdp + 7));
+ rte_compiler_barrier();
+ const __m128i raw_desc6 =
+ _mm_load_si128((void *)(rxdp + 6));
+ rte_compiler_barrier();
+ const __m128i raw_desc5 =
+ _mm_load_si128((void *)(rxdp + 5));
+ rte_compiler_barrier();
+ const __m128i raw_desc4 =
+ _mm_load_si128((void *)(rxdp + 4));
+ rte_compiler_barrier();
+ const __m128i raw_desc3 =
+ _mm_load_si128((void *)(rxdp + 3));
+ rte_compiler_barrier();
+ const __m128i raw_desc2 =
+ _mm_load_si128((void *)(rxdp + 2));
+ rte_compiler_barrier();
+ const __m128i raw_desc1 =
+ _mm_load_si128((void *)(rxdp + 1));
+ rte_compiler_barrier();
+ const __m128i raw_desc0 =
+ _mm_load_si128((void *)(rxdp + 0));
+
+ raw_desc4_7 = _mm512_broadcast_i32x4(raw_desc4);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc5, 1);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc6, 2);
+ raw_desc4_7 = _mm512_inserti32x4(raw_desc4_7, raw_desc7, 3);
+ raw_desc0_3 = _mm512_broadcast_i32x4(raw_desc0);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc1, 1);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc2, 2);
+ raw_desc0_3 = _mm512_inserti32x4(raw_desc0_3, raw_desc3, 3);
+
+ /**
+ * convert descriptors 4-7 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len4_7 = _mm512_slli_epi32(raw_desc4_7,
+ PKTLEN_SHIFT);
+ const __m512i desc4_7 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc4_7,
+ len4_7);
+ __m512i mb4_7 = _mm512_shuffle_epi8(desc4_7, shuf_msk);
+
+ /**
+ * to get packet types, shift 64-bit values down 30 bits
+ * and so ptype is in lower 8-bits in each
+ */
+ const __m512i ptypes4_7 = _mm512_srli_epi64(desc4_7, 16);
+ const __m256i ptypes6_7 = _mm512_extracti64x4_epi64(ptypes4_7, 1);
+ const __m256i ptypes4_5 = _mm512_extracti64x4_epi64(ptypes4_7, 0);
+ const uint8_t ptype7 = _mm256_extract_epi8(ptypes6_7, 16);
+ const uint8_t ptype6 = _mm256_extract_epi8(ptypes6_7, 0);
+ const uint8_t ptype5 = _mm256_extract_epi8(ptypes4_5, 16);
+ const uint8_t ptype4 = _mm256_extract_epi8(ptypes4_5, 0);
+
+ const __m512i ptype4_7 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype7],
+ 0, 0, 0, type_table[ptype6],
+ 0, 0, 0, type_table[ptype5],
+ 0, 0, 0, type_table[ptype4]);
+ mb4_7 = _mm512_mask_blend_epi32(0x1111, mb4_7, ptype4_7);
+
+ /**
+ * convert descriptors 0-3 into mbufs, adjusting length and
+ * re-arranging fields. Then write into the mbuf
+ */
+ const __m512i len0_3 = _mm512_slli_epi32(raw_desc0_3,
+ PKTLEN_SHIFT);
+ const __m512i desc0_3 = _mm512_mask_blend_epi16(IDPF_RX_LEN_MASK,
+ raw_desc0_3,
+ len0_3);
+ __m512i mb0_3 = _mm512_shuffle_epi8(desc0_3, shuf_msk);
+
+ /* get the packet types */
+ const __m512i ptypes0_3 = _mm512_srli_epi64(desc0_3, 16);
+ const __m256i ptypes2_3 = _mm512_extracti64x4_epi64(ptypes0_3, 1);
+ const __m256i ptypes0_1 = _mm512_extracti64x4_epi64(ptypes0_3, 0);
+ const uint8_t ptype3 = _mm256_extract_epi8(ptypes2_3, 16);
+ const uint8_t ptype2 = _mm256_extract_epi8(ptypes2_3, 0);
+ const uint8_t ptype1 = _mm256_extract_epi8(ptypes0_1, 16);
+ const uint8_t ptype0 = _mm256_extract_epi8(ptypes0_1, 0);
+
+ const __m512i ptype0_3 = _mm512_set_epi32
+ (0, 0, 0, type_table[ptype3],
+ 0, 0, 0, type_table[ptype2],
+ 0, 0, 0, type_table[ptype1],
+ 0, 0, 0, type_table[ptype0]);
+ mb0_3 = _mm512_mask_blend_epi32(0x1111, mb0_3, ptype0_3);
+
+ /**
+ * use permute/extract to get status content
+ * After the operations, the packets status flags are in the
+ * order (hi->lo): [1, 3, 5, 7, 0, 2, 4, 6]
+ */
+ /* merge the status bits into one register */
+ const __m512i status_permute_msk = _mm512_set_epi32
+ (0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 22, 30, 6, 14,
+ 18, 26, 2, 10);
+ const __m512i raw_status0_7 = _mm512_permutex2var_epi32
+ (raw_desc4_7, status_permute_msk, raw_desc0_3);
+ __m256i status0_7 = _mm512_extracti64x4_epi64
+ (raw_status0_7, 0);
+
+ /* now do flag manipulation */
+
+ /**
+ * At this point, we have the 8 sets of flags in the low 16-bits
+ * of each 32-bit value.
+ * We want to extract these, and merge them with the mbuf init
+ * data so we can do a single write to the mbuf to set the flags
+ * and all the other initialization fields. Extracting the
+ * appropriate flags means that we have to do a shift and blend
+ * for each mbuf before we do the write. However, we can also
+ * add in the previously computed rx_descriptor fields to
+ * make a single 256-bit write per mbuf
+ */
+ /* check the structure matches expectations */
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf,
+ rearm_data),
+ 16));
+ /* build up data and do writes */
+ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5,
+ rearm6, rearm7;
+ const __m256i mb4_5 = _mm512_extracti64x4_epi64(mb4_7, 0);
+ const __m256i mb6_7 = _mm512_extracti64x4_epi64(mb4_7, 1);
+ const __m256i mb0_1 = _mm512_extracti64x4_epi64(mb0_3, 0);
+ const __m256i mb2_3 = _mm512_extracti64x4_epi64(mb0_3, 1);
+
+ rearm6 = _mm256_permute2f128_si256(mbuf_init, mb6_7, 0x20);
+ rearm4 = _mm256_permute2f128_si256(mbuf_init, mb4_5, 0x20);
+ rearm2 = _mm256_permute2f128_si256(mbuf_init, mb2_3, 0x20);
+ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20);
+
+ /* write to mbuf */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data,
+ rearm6);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data,
+ rearm4);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data,
+ rearm2);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data,
+ rearm0);
+
+ rearm7 = _mm256_blend_epi32(mbuf_init, mb6_7, 0xF0);
+ rearm5 = _mm256_blend_epi32(mbuf_init, mb4_5, 0xF0);
+ rearm3 = _mm256_blend_epi32(mbuf_init, mb2_3, 0xF0);
+ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0);
+
+ /* again write to mbufs */
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data,
+ rearm7);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data,
+ rearm5);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data,
+ rearm3);
+ _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data,
+ rearm1);
+
+ /* perform dd_check */
+ status0_7 = _mm256_and_si256(status0_7, dd_check);
+ status0_7 = _mm256_packs_epi32(status0_7,
+ _mm256_setzero_si256());
+
+ uint64_t burst = __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_extracti128_si256
+ (status0_7, 1)));
+ burst += __builtin_popcountll
+ (_mm_cvtsi128_si64
+ (_mm256_castsi256_si128(status0_7)));
+ received += burst;
+ if (burst != IDPF_DESCS_PER_LOOP_AVX)
+ break;
+ }
+
+ /* update tail pointers */
+ rxq->rx_tail += received;
+ rxq->rx_tail &= (rxq->nb_rx_desc - 1);
+ if ((rxq->rx_tail & 1) == 1 && received > 1) { /* keep aligned */
+ rxq->rx_tail--;
+ received--;
+ }
+ rxq->rxrearm_nb += received;
+ return received;
+}
+
+/**
+ * Notice:
+ * - nb_pkts < IDPF_DESCS_PER_LOOP, just return no packet
+ */
+uint16_t
+idpf_singleq_recv_pkts_avx512(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ return _idpf_singleq_recv_raw_pkts_avx512(rx_queue, rx_pkts, nb_pkts);
+}
+
+static __rte_always_inline int
+idpf_tx_free_bufs_avx512(struct idpf_tx_queue *txq)
+{
+ struct idpf_tx_vec_entry *txep;
+ uint32_t n;
+ uint32_t i;
+ int nb_free = 0;
+ struct rte_mbuf *m, *free[txq->rs_thresh];
+
+ /* check DD bits on threshold descriptor */
+ if ((txq->tx_ring[txq->next_dd].qw1.cmd_dtype &
+ rte_cpu_to_le_64(IDPF_TXD_QW1_DTYPE_M)) !=
+ rte_cpu_to_le_64(IDPF_TX_DESC_DTYPE_DESC_DONE))
+ return 0;
+
+ n = txq->rs_thresh;
+
+ /* first buffer to free from S/W ring is at index
+ * tx_next_dd - (tx_rs_thresh-1)
+ */
+ txep = (void *)txq->sw_ring;
+ txep += txq->next_dd - (n - 1);
+
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE && (n & 31) == 0) {
+ struct rte_mempool *mp = txep[0].mbuf->pool;
+ struct rte_mempool_cache *cache = rte_mempool_default_cache(mp,
+ rte_lcore_id());
+ void **cache_objs;
+
+ if (cache == NULL || cache->len == 0)
+ goto normal;
+
+ cache_objs = &cache->objs[cache->len];
+
+ if (n > RTE_MEMPOOL_CACHE_MAX_SIZE) {
+ rte_mempool_ops_enqueue_bulk(mp, (void *)txep, n);
+ goto done;
+ }
+
+ /* The cache follows the following algorithm
+ * 1. Add the objects to the cache
+ * 2. Anything greater than the cache min value (if it crosses the
+ * cache flush threshold) is flushed to the ring.
+ */
+ /* Add elements back into the cache */
+ uint32_t copied = 0;
+ /* n is multiple of 32 */
+ while (copied < n) {
+ const __m512i a = _mm512_loadu_si512(&txep[copied]);
+ const __m512i b = _mm512_loadu_si512(&txep[copied + 8]);
+ const __m512i c = _mm512_loadu_si512(&txep[copied + 16]);
+ const __m512i d = _mm512_loadu_si512(&txep[copied + 24]);
+
+ _mm512_storeu_si512(&cache_objs[copied], a);
+ _mm512_storeu_si512(&cache_objs[copied + 8], b);
+ _mm512_storeu_si512(&cache_objs[copied + 16], c);
+ _mm512_storeu_si512(&cache_objs[copied + 24], d);
+ copied += 32;
+ }
+ cache->len += n;
+
+ if (cache->len >= cache->flushthresh) {
+ rte_mempool_ops_enqueue_bulk(mp,
+ &cache->objs[cache->size],
+ cache->len - cache->size);
+ cache->len = cache->size;
+ }
+ goto done;
+ }
+
+normal:
+ m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+ if (likely(m != NULL)) {
+ free[0] = m;
+ nb_free = 1;
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (likely(m != NULL)) {
+ if (likely(m->pool == free[0]->pool)) {
+ free[nb_free++] = m;
+ } else {
+ rte_mempool_put_bulk(free[0]->pool,
+ (void *)free,
+ nb_free);
+ free[0] = m;
+ nb_free = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free);
+ } else {
+ for (i = 1; i < n; i++) {
+ m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+ if (m != NULL)
+ rte_mempool_put(m->pool, m);
+ }
+ }
+
+done:
+ /* buffers were freed, update counters */
+ txq->nb_free = (uint16_t)(txq->nb_free + txq->rs_thresh);
+ txq->next_dd = (uint16_t)(txq->next_dd + txq->rs_thresh);
+ if (txq->next_dd >= txq->nb_tx_desc)
+ txq->next_dd = (uint16_t)(txq->rs_thresh - 1);
+
+ return txq->rs_thresh;
+}
+
+static __rte_always_inline void
+tx_backlog_entry_avx512(struct idpf_tx_vec_entry *txep,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i;
+
+ for (i = 0; i < (int)nb_pkts; ++i)
+ txep[i].mbuf = tx_pkts[i];
+}
+
+#define IDPF_FLEX_TXD_QW1_BUF_SZ_S 48
+static __rte_always_inline void
+idpf_vtx1(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf *pkt, uint64_t flags)
+{
+ uint64_t high_qw =
+ (IDPF_TX_DESC_DTYPE_FLEX_DATA << IDPF_FLEX_TXD_QW1_DTYPE_S |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S) |
+ ((uint64_t)pkt->data_len << IDPF_FLEX_TXD_QW1_BUF_SZ_S));
+
+ __m128i descriptor = _mm_set_epi64x(high_qw,
+ pkt->buf_iova + pkt->data_off);
+ _mm_storeu_si128((__m128i *)txdp, descriptor);
+}
+
+#define IDPF_TX_LEN_MASK 0xAA
+#define IDPF_TX_OFF_MASK 0x55
+static __rte_always_inline void
+idpf_vtx(volatile struct idpf_flex_tx_desc *txdp,
+ struct rte_mbuf **pkt, uint16_t nb_pkts, uint64_t flags)
+{
+ const uint64_t hi_qw_tmpl = (IDPF_TX_DESC_DTYPE_FLEX_DATA |
+ ((uint64_t)flags << IDPF_FLEX_TXD_QW1_CMD_S));
+
+ /* if unaligned on 32-bit boundary, do one to align */
+ if (((uintptr_t)txdp & 0x1F) != 0 && nb_pkts != 0) {
+ idpf_vtx1(txdp, *pkt, flags);
+ nb_pkts--, txdp++, pkt++;
+ }
+
+ /* do 4 at a time while possible, in bursts */
+ for (; nb_pkts > 3; txdp += 4, pkt += 4, nb_pkts -= 4) {
+ uint64_t hi_qw3 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[3]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw2 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[2]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw1 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[1]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+ uint64_t hi_qw0 =
+ hi_qw_tmpl |
+ ((uint64_t)pkt[0]->data_len <<
+ IDPF_FLEX_TXD_QW1_BUF_SZ_S);
+
+ __m512i desc0_3 =
+ _mm512_set_epi64
+ (hi_qw3,
+ pkt[3]->buf_iova + pkt[3]->data_off,
+ hi_qw2,
+ pkt[2]->buf_iova + pkt[2]->data_off,
+ hi_qw1,
+ pkt[1]->buf_iova + pkt[1]->data_off,
+ hi_qw0,
+ pkt[0]->buf_iova + pkt[0]->data_off);
+ _mm512_storeu_si512((void *)txdp, desc0_3);
+ }
+
+ /* do any last ones */
+ while (nb_pkts) {
+ idpf_vtx1(txdp, *pkt, flags);
+ txdp++, pkt++, nb_pkts--;
+ }
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_fixed_burst_vec_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct idpf_tx_queue *txq = tx_queue;
+ volatile struct idpf_flex_tx_desc *txdp;
+ struct idpf_tx_vec_entry *txep;
+ uint16_t n, nb_commit, tx_id;
+ uint64_t flags = IDPF_TX_FLEX_DESC_CMD_EOP;
+ uint64_t rs = IDPF_TX_FLEX_DESC_CMD_RS | flags;
+
+ /* cross rx_thresh boundary is not allowed */
+ nb_pkts = RTE_MIN(nb_pkts, txq->rs_thresh);
+
+ if (txq->nb_free < txq->free_thresh)
+ idpf_tx_free_bufs_avx512(txq);
+
+ nb_pkts = (uint16_t)RTE_MIN(txq->nb_free, nb_pkts);
+ nb_commit = nb_pkts;
+ if (unlikely(nb_pkts == 0))
+ return 0;
+
+ tx_id = txq->tx_tail;
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+
+ txq->nb_free = (uint16_t)(txq->nb_free - nb_pkts);
+
+ n = (uint16_t)(txq->nb_tx_desc - tx_id);
+ if (nb_commit >= n) {
+ tx_backlog_entry_avx512(txep, tx_pkts, n);
+
+ idpf_vtx(txdp, tx_pkts, n - 1, flags);
+ tx_pkts += (n - 1);
+ txdp += (n - 1);
+
+ idpf_vtx1(txdp, *tx_pkts++, rs);
+
+ nb_commit = (uint16_t)(nb_commit - n);
+
+ tx_id = 0;
+ txq->next_rs = (uint16_t)(txq->rs_thresh - 1);
+
+ /* avoid reach the end of ring */
+ txdp = &txq->tx_ring[tx_id];
+ txep = (void *)txq->sw_ring;
+ txep += tx_id;
+ }
+
+ tx_backlog_entry_avx512(txep, tx_pkts, nb_commit);
+
+ idpf_vtx(txdp, tx_pkts, nb_commit, flags);
+
+ tx_id = (uint16_t)(tx_id + nb_commit);
+ if (tx_id > txq->next_rs) {
+ txq->tx_ring[txq->next_rs].qw1.cmd_dtype |=
+ rte_cpu_to_le_64(((uint64_t)IDPF_TX_FLEX_DESC_CMD_RS) <<
+ IDPF_FLEX_TXD_QW1_CMD_S);
+ txq->next_rs =
+ (uint16_t)(txq->next_rs + txq->rs_thresh);
+ }
+
+ txq->tx_tail = tx_id;
+
+ IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
+
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+idpf_xmit_pkts_vec_avx512_cmn(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ uint16_t nb_tx = 0;
+ struct idpf_tx_queue *txq = tx_queue;
+
+ while (nb_pkts) {
+ uint16_t ret, num;
+
+ num = (uint16_t)RTE_MIN(nb_pkts, txq->rs_thresh);
+ ret = idpf_xmit_fixed_burst_vec_avx512(tx_queue, &tx_pkts[nb_tx],
+ num);
+ nb_tx += ret;
+ nb_pkts -= ret;
+ if (ret < num)
+ break;
+ }
+
+ return nb_tx;
+}
+
+uint16_t
+idpf_singleq_xmit_pkts_avx512(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ return idpf_xmit_pkts_vec_avx512_cmn(tx_queue, tx_pkts, nb_pkts);
+}
+
+static inline void
+idpf_singleq_tx_release_mbufs_avx512(struct idpf_tx_queue *txq)
+{
+ unsigned int i;
+ const uint16_t max_desc = (uint16_t)(txq->nb_tx_desc - 1);
+ struct idpf_tx_vec_entry *swr = (void *)txq->sw_ring;
+
+ if (txq->sw_ring == NULL || txq->nb_free == max_desc)
+ return;
+
+ i = txq->next_dd - txq->rs_thresh + 1;
+ if (txq->tx_tail < i) {
+ for (; i < txq->nb_tx_desc; i++) {
+ rte_pktmbuf_free_seg(swr[i].mbuf);
+ swr[i].mbuf = NULL;
+ }
+ i = 0;
+ }
+}
+
+static const struct idpf_txq_ops avx512_singleq_tx_vec_ops = {
+ .release_mbufs = idpf_singleq_tx_release_mbufs_avx512,
+};
+
+int __rte_cold
+idpf_singleq_tx_vec_setup_avx512(struct idpf_tx_queue *txq)
+{
+ txq->ops = &avx512_singleq_tx_vec_ops;
+ return 0;
+}
diff --git a/drivers/net/idpf/idpf_rxtx_vec_common.h b/drivers/net/idpf/idpf_rxtx_vec_common.h
new file mode 100644
index 0000000000..0f4e10e154
--- /dev/null
+++ b/drivers/net/idpf/idpf_rxtx_vec_common.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
+
+#ifndef _IDPF_RXTX_VEC_COMMON_H_
+#define _IDPF_RXTX_VEC_COMMON_H_
+#include <stdint.h>
+#include <ethdev_driver.h>
+#include <rte_malloc.h>
+
+#include "idpf_ethdev.h"
+#include "idpf_rxtx.h"
+
+#ifndef __INTEL_COMPILER
+#pragma GCC diagnostic ignored "-Wcast-qual"
+#endif
+
+#define IDPF_VECTOR_PATH 0
+#define ICE_RX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+#define ICE_TX_NO_VECTOR_FLAGS ( \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS)
+
+static inline int
+idpf_rx_vec_queue_default(struct idpf_rx_queue *rxq)
+{
+ if (rxq == NULL)
+ return -1;
+
+ if (rte_is_power_of_2(rxq->nb_rx_desc) == 0)
+ return -1;
+
+ if (rxq->rx_free_thresh < IDPF_VPMD_RX_MAX_BURST)
+ return -1;
+
+ if ((rxq->nb_rx_desc % rxq->rx_free_thresh) != 0)
+ return -1;
+
+ if ((rxq->offloads & ICE_RX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_queue_default(struct idpf_tx_queue *txq)
+{
+ if (txq == NULL)
+ return -1;
+
+ if (txq->rs_thresh < IDPF_VPMD_TX_MAX_BURST ||
+ (txq->rs_thresh & 3) != 0)
+ return -1;
+
+ if ((txq->offloads & ICE_TX_NO_VECTOR_FLAGS) != 0)
+ return -1;
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_rx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_rx_queue *rxq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ rxq = dev->data->rx_queues[i];
+ ret = (idpf_rx_vec_queue_default(rxq));
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+static inline int
+idpf_tx_vec_dev_check_default(struct rte_eth_dev *dev)
+{
+ int i;
+ struct idpf_tx_queue *txq;
+ int ret = 0;
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ txq = dev->data->tx_queues[i];
+ ret = idpf_tx_vec_queue_default(txq);
+ if (ret < 0)
+ return -1;
+ }
+
+ return IDPF_VECTOR_PATH;
+}
+
+#endif /*_IDPF_RXTX_VEC_COMMON_H_*/
diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
index b632b76656..da99c098ab 100644
--- a/drivers/net/idpf/meson.build
+++ b/drivers/net/idpf/meson.build
@@ -14,3 +14,31 @@ sources = files(
'idpf_rxtx.c',
'idpf_vchnl.c',
)
+
+if arch_subdir == 'x86'
+ idpf_avx512_cpu_support = (
+ cc.get_define('__AVX512F__', args: machine_args) != '' and
+ cc.get_define('__AVX512BW__', args: machine_args) != ''
+ )
+
+ idpf_avx512_cc_support = (
+ not machine_args.contains('-mno-avx512f') and
+ cc.has_argument('-mavx512f') and
+ cc.has_argument('-mavx512bw')
+ )
+
+ if idpf_avx512_cpu_support == true or idpf_avx512_cc_support == true
+ cflags += ['-DCC_AVX512_SUPPORT']
+ avx512_args = [cflags, '-mavx512f', '-mavx512bw']
+ if cc.has_argument('-march=skylake-avx512')
+ avx512_args += '-march=skylake-avx512'
+ endif
+ idpf_avx512_lib = static_library('idpf_avx512_lib',
+ 'idpf_rxtx_vec_avx512.c',
+ dependencies: [static_rte_common_idpf, static_rte_ethdev, static_rte_bus_pci,
+ static_rte_kvargs, static_rte_hash],
+ include_directories: includes,
+ c_args: avx512_args)
+ objs += idpf_avx512_lib.extract_objects('idpf_rxtx_vec_avx512.c')
+ endif
+endif
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* [PATCH v18 18/18] net/idpf: add support for timestamp offload
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (16 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
@ 2022-10-31 8:33 ` beilei.xing
2022-10-31 13:38 ` [PATCH v18 00/18] add support for idpf PMD in DPDK Thomas Monjalon
18 siblings, 0 replies; 376+ messages in thread
From: beilei.xing @ 2022-10-31 8:33 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, Junfeng Guo, Wenjing Qiao
From: Junfeng Guo <junfeng.guo@intel.com>
Add support for timestamp offload.
Signed-off-by: Wenjing Qiao <wenjing.qiao@intel.com>
Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
---
doc/guides/nics/features/idpf.ini | 1 +
drivers/net/idpf/idpf_ethdev.c | 5 +-
drivers/net/idpf/idpf_ethdev.h | 3 ++
drivers/net/idpf/idpf_rxtx.c | 65 ++++++++++++++++++++++
drivers/net/idpf/idpf_rxtx.h | 90 +++++++++++++++++++++++++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/features/idpf.ini b/doc/guides/nics/features/idpf.ini
index d82b4aa0ff..099fd7f216 100644
--- a/doc/guides/nics/features/idpf.ini
+++ b/doc/guides/nics/features/idpf.ini
@@ -11,6 +11,7 @@ MTU update = Y
TSO = P
L3 checksum offload = P
L4 checksum offload = P
+Timestamp offload = P
Linux = Y
x86-32 = Y
x86-64 = Y
diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index cd4ebcc2c6..50aac65daf 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -22,6 +22,8 @@ rte_spinlock_t idpf_adapter_lock;
struct idpf_adapter_list idpf_adapter_list;
bool idpf_adapter_list_init;
+uint64_t idpf_timestamp_dynflag;
+
static const char * const idpf_valid_args[] = {
IDPF_TX_SINGLE_Q,
IDPF_RX_SINGLE_Q,
@@ -65,7 +67,8 @@ idpf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
- RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM;
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP;
dev_info->tx_offload_capa =
RTE_ETH_TX_OFFLOAD_TCP_TSO |
diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
index 7d54e5db60..ccdf4abe40 100644
--- a/drivers/net/idpf/idpf_ethdev.h
+++ b/drivers/net/idpf/idpf_ethdev.h
@@ -167,6 +167,9 @@ struct idpf_adapter {
bool tx_vec_allowed;
bool rx_use_avx512;
bool tx_use_avx512;
+
+ /* For PTP */
+ uint64_t time_hw;
};
TAILQ_HEAD(idpf_adapter_list, idpf_adapter);
diff --git a/drivers/net/idpf/idpf_rxtx.c b/drivers/net/idpf/idpf_rxtx.c
index 9e20f2b9d3..bafa007faf 100644
--- a/drivers/net/idpf/idpf_rxtx.c
+++ b/drivers/net/idpf/idpf_rxtx.c
@@ -10,6 +10,8 @@
#include "idpf_rxtx.h"
#include "idpf_rxtx_vec_common.h"
+static int idpf_timestamp_dynfield_offset = -1;
+
static int
check_rx_thresh(uint16_t nb_desc, uint16_t thresh)
{
@@ -900,6 +902,24 @@ idpf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
return idpf_tx_split_queue_setup(dev, queue_idx, nb_desc,
socket_id, tx_conf);
}
+
+static int
+idpf_register_ts_mbuf(struct idpf_rx_queue *rxq)
+{
+ int err;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* Register mbuf field and flag for Rx timestamp */
+ err = rte_mbuf_dyn_rx_timestamp_register(&idpf_timestamp_dynfield_offset,
+ &idpf_timestamp_dynflag);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR,
+ "Cannot register mbuf field/flag for timestamp");
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int
idpf_alloc_single_rxq_mbufs(struct idpf_rx_queue *rxq)
{
@@ -993,6 +1013,13 @@ idpf_rx_queue_init(struct rte_eth_dev *dev, uint16_t rx_queue_id)
return -EINVAL;
}
+ err = idpf_register_ts_mbuf(rxq);
+ if (err != 0) {
+ PMD_DRV_LOG(ERR, "fail to regidter timestamp mbuf %u",
+ rx_queue_id);
+ return -EIO;
+ }
+
if (rxq->bufq1 == NULL) {
/* Single queue */
err = idpf_alloc_single_rxq_mbufs(rxq);
@@ -1354,6 +1381,7 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
struct idpf_rx_queue *rxq;
const uint32_t *ptype_tbl;
uint8_t status_err0_qw1;
+ struct idpf_adapter *ad;
struct rte_mbuf *rxm;
uint16_t rx_id_bufq1;
uint16_t rx_id_bufq2;
@@ -1363,9 +1391,11 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t gen_id;
uint16_t rx_id;
uint16_t nb_rx;
+ uint64_t ts_ns;
nb_rx = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1376,6 +1406,9 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_desc_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rx_desc = &rx_desc_ring[rx_id];
@@ -1431,6 +1464,18 @@ idpf_splitq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
status_err0_qw1 = rx_desc->status_err0_qw1;
pkt_flags = idpf_splitq_rx_csum_offload(status_err0_qw1);
pkt_flags |= idpf_splitq_rx_rss_offload(rxm, rx_desc);
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rx_desc->ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
rxm->ol_flags |= pkt_flags;
@@ -1732,18 +1777,22 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
const uint32_t *ptype_tbl;
uint16_t rx_id, nb_hold;
struct rte_eth_dev *dev;
+ struct idpf_adapter *ad;
uint16_t rx_packet_len;
struct rte_mbuf *rxm;
struct rte_mbuf *nmb;
uint16_t rx_status0;
uint64_t pkt_flags;
uint64_t dma_addr;
+ uint64_t ts_ns;
uint16_t nb_rx;
nb_rx = 0;
nb_hold = 0;
rxq = rx_queue;
+ ad = rxq->adapter;
+
if (unlikely(rxq == NULL) || unlikely(!rxq->q_started))
return nb_rx;
@@ -1751,6 +1800,9 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rx_ring = rxq->rx_ring;
ptype_tbl = rxq->adapter->ptype_tbl;
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0)
+ rxq->hw_register_set = 1;
+
while (nb_rx < nb_pkts) {
rxdp = &rx_ring[rx_id];
rx_status0 = rte_le_to_cpu_16(rxdp->flex_nic_wb.status_error0);
@@ -1810,6 +1862,19 @@ idpf_singleq_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
rxm->ol_flags |= pkt_flags;
+ if (idpf_timestamp_dynflag > 0 &&
+ (rxq->offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP) != 0) {
+ /* timestamp */
+ ts_ns = idpf_tstamp_convert_32b_64b(ad,
+ rxq->hw_register_set,
+ rte_le_to_cpu_32(rxd.flex_nic_wb.flex_ts.ts_high));
+ rxq->hw_register_set = 0;
+ *RTE_MBUF_DYNFIELD(rxm,
+ idpf_timestamp_dynfield_offset,
+ rte_mbuf_timestamp_t *) = ts_ns;
+ rxm->ol_flags |= idpf_timestamp_dynflag;
+ }
+
rx_pkts[nb_rx++] = rxm;
}
rxq->rx_tail = rx_id;
diff --git a/drivers/net/idpf/idpf_rxtx.h b/drivers/net/idpf/idpf_rxtx.h
index 506bf8a57d..a98f0285c8 100644
--- a/drivers/net/idpf/idpf_rxtx.h
+++ b/drivers/net/idpf/idpf_rxtx.h
@@ -7,6 +7,41 @@
#include "idpf_ethdev.h"
+/* MTS */
+#define GLTSYN_CMD_SYNC_0_0 (PF_TIMESYNC_BASE + 0x0)
+#define PF_GLTSYN_SHTIME_0_0 (PF_TIMESYNC_BASE + 0x4)
+#define PF_GLTSYN_SHTIME_L_0 (PF_TIMESYNC_BASE + 0x8)
+#define PF_GLTSYN_SHTIME_H_0 (PF_TIMESYNC_BASE + 0xC)
+#define GLTSYN_ART_L_0 (PF_TIMESYNC_BASE + 0x10)
+#define GLTSYN_ART_H_0 (PF_TIMESYNC_BASE + 0x14)
+#define PF_GLTSYN_SHTIME_0_1 (PF_TIMESYNC_BASE + 0x24)
+#define PF_GLTSYN_SHTIME_L_1 (PF_TIMESYNC_BASE + 0x28)
+#define PF_GLTSYN_SHTIME_H_1 (PF_TIMESYNC_BASE + 0x2C)
+#define PF_GLTSYN_SHTIME_0_2 (PF_TIMESYNC_BASE + 0x44)
+#define PF_GLTSYN_SHTIME_L_2 (PF_TIMESYNC_BASE + 0x48)
+#define PF_GLTSYN_SHTIME_H_2 (PF_TIMESYNC_BASE + 0x4C)
+#define PF_GLTSYN_SHTIME_0_3 (PF_TIMESYNC_BASE + 0x64)
+#define PF_GLTSYN_SHTIME_L_3 (PF_TIMESYNC_BASE + 0x68)
+#define PF_GLTSYN_SHTIME_H_3 (PF_TIMESYNC_BASE + 0x6C)
+
+#define PF_TIMESYNC_BAR4_BASE 0x0E400000
+#define GLTSYN_ENA (PF_TIMESYNC_BAR4_BASE + 0x90)
+#define GLTSYN_CMD (PF_TIMESYNC_BAR4_BASE + 0x94)
+#define GLTSYC_TIME_L (PF_TIMESYNC_BAR4_BASE + 0x104)
+#define GLTSYC_TIME_H (PF_TIMESYNC_BAR4_BASE + 0x108)
+
+#define GLTSYN_CMD_SYNC_0_4 (PF_TIMESYNC_BAR4_BASE + 0x110)
+#define PF_GLTSYN_SHTIME_L_4 (PF_TIMESYNC_BAR4_BASE + 0x118)
+#define PF_GLTSYN_SHTIME_H_4 (PF_TIMESYNC_BAR4_BASE + 0x11C)
+#define GLTSYN_INCVAL_L (PF_TIMESYNC_BAR4_BASE + 0x150)
+#define GLTSYN_INCVAL_H (PF_TIMESYNC_BAR4_BASE + 0x154)
+#define GLTSYN_SHADJ_L (PF_TIMESYNC_BAR4_BASE + 0x158)
+#define GLTSYN_SHADJ_H (PF_TIMESYNC_BAR4_BASE + 0x15C)
+
+#define GLTSYN_CMD_SYNC_0_5 (PF_TIMESYNC_BAR4_BASE + 0x130)
+#define PF_GLTSYN_SHTIME_L_5 (PF_TIMESYNC_BAR4_BASE + 0x138)
+#define PF_GLTSYN_SHTIME_H_5 (PF_TIMESYNC_BAR4_BASE + 0x13C)
+
/* In QLEN must be whole number of 32 descriptors. */
#define IDPF_ALIGN_RING_DESC 32
#define IDPF_MIN_RING_DESC 32
@@ -43,6 +78,8 @@
(sizeof(struct virtchnl2_ptype) + \
(((p)->proto_id_count ? ((p)->proto_id_count - 1) : 0) * sizeof((p)->proto_id[0])))
+extern uint64_t idpf_timestamp_dynflag;
+
struct idpf_rx_queue {
struct idpf_adapter *adapter; /* the adapter this queue belongs to */
struct rte_mempool *mp; /* mbuf pool to populate Rx ring */
@@ -198,4 +235,57 @@ void idpf_stop_queues(struct rte_eth_dev *dev);
void idpf_set_rx_function(struct rte_eth_dev *dev);
void idpf_set_tx_function(struct rte_eth_dev *dev);
+#define IDPF_TIMESYNC_REG_WRAP_GUARD_BAND 10000
+/* Helper function to convert a 32b nanoseconds timestamp to 64b. */
+static inline uint64_t
+
+idpf_tstamp_convert_32b_64b(struct idpf_adapter *ad, uint32_t flag,
+ uint32_t in_timestamp)
+{
+#ifdef RTE_ARCH_X86_64
+ struct idpf_hw *hw = &ad->hw;
+ const uint64_t mask = 0xFFFFFFFF;
+ uint32_t hi, lo, lo2, delta;
+ uint64_t ns;
+
+ if (flag != 0) {
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ IDPF_WRITE_REG(hw, GLTSYN_CMD_SYNC_0_0, PF_GLTSYN_CMD_SYNC_EXEC_CMD_M |
+ PF_GLTSYN_CMD_SYNC_SHTIME_EN_M);
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ /*
+ * On typical system, the delta between lo and lo2 is ~1000ns,
+ * so 10000 seems a large-enough but not overly-big guard band.
+ */
+ if (lo > (UINT32_MAX - IDPF_TIMESYNC_REG_WRAP_GUARD_BAND))
+ lo2 = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ else
+ lo2 = lo;
+
+ if (lo2 < lo) {
+ lo = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_L_0);
+ hi = IDPF_READ_REG(hw, PF_GLTSYN_SHTIME_H_0);
+ }
+
+ ad->time_hw = ((uint64_t)hi << 32) | lo;
+ }
+
+ delta = (in_timestamp - (uint32_t)(ad->time_hw & mask));
+ if (delta > (mask / 2)) {
+ delta = ((uint32_t)(ad->time_hw & mask) - in_timestamp);
+ ns = ad->time_hw - delta;
+ } else {
+ ns = ad->time_hw + delta;
+ }
+
+ return ns;
+#else /* !RTE_ARCH_X86_64 */
+ RTE_SET_USED(ad);
+ RTE_SET_USED(flag);
+ RTE_SET_USED(in_timestamp);
+ return 0;
+#endif /* RTE_ARCH_X86_64 */
+}
+
#endif /* _IDPF_RXTX_H_ */
--
2.26.2
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH v18 00/18] add support for idpf PMD in DPDK
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
` (17 preceding siblings ...)
2022-10-31 8:33 ` [PATCH v18 18/18] net/idpf: add support for timestamp offload beilei.xing
@ 2022-10-31 13:38 ` Thomas Monjalon
18 siblings, 0 replies; 376+ messages in thread
From: Thomas Monjalon @ 2022-10-31 13:38 UTC (permalink / raw)
To: jingjing.wu, beilei.xing; +Cc: dev, qi.z.zhang, helin.zhang, andrew.rybchenko
31/10/2022 09:33, beilei.xing@intel.com:
> From: Beilei Xing <beilei.xing@intel.com>
>
> This patchset introduced the idpf (Infrastructure Data Path Function) PMD in DPDK for Intel® IPU E2000 (Device ID: 0x1452).
> The Intel® IPU E2000 targets to deliver high performance under real workloads with security and isolation.
> Please refer to
> https://www.intel.com/content/www/us/en/products/network-io/infrastructure-processing-units/asic/e2000-asic.html
> for more information.
>
> Linux upstream is still ongoing, previous work refers to https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20220128001009.721392-20-alan.brady@intel.com/.
I've fixed/improved doc and build files.
Applied, thanks.
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 03/13] net/idpf: support device initialization
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
2022-08-03 15:11 ` Stephen Hemminger
@ 2022-10-31 18:00 ` Ali Alnubani
2022-11-01 6:55 ` Xing, Beilei
2022-11-02 15:31 ` Raslan Darawsheh
2 siblings, 1 reply; 376+ messages in thread
From: Ali Alnubani @ 2022-10-31 18:00 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang, NBU-Contact-Thomas Monjalon (EXTERNAL)
> -----Original Message-----
> From: Junfeng Guo <junfeng.guo@intel.com>
> Sent: Wednesday, August 3, 2022 2:31 PM
> To: qi.z.zhang@intel.com; jingjing.wu@intel.com; beilei.xing@intel.com
> Cc: dev@dpdk.org; junfeng.guo@intel.com; Xiaoyun Li
> <xiaoyun.li@intel.com>; Xiao Wang <xiao.w.wang@intel.com>
> Subject: [PATCH 03/13] net/idpf: support device initialization
>
> Support device init and the following dev ops:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
Hello,
This patch is causing the following build failure in latest main (6a88cbc) with clang 3.4.2 in CentOS 7:
drivers/net/idpf/idpf_vchnl.c:141:13: error: comparison of constant 522 with expression of type 'enum virtchnl_ops' is always false [-Werror,-Wtautological-constant-out-of-range-compare]
Regards,
Ali
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 03/13] net/idpf: support device initialization
2022-10-31 18:00 ` Ali Alnubani
@ 2022-11-01 6:55 ` Xing, Beilei
0 siblings, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-11-01 6:55 UTC (permalink / raw)
To: Ali Alnubani, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun, Wang, Xiao W, NBU-Contact-Thomas Monjalon (EXTERNAL)
> -----Original Message-----
> From: Ali Alnubani <alialnu@nvidia.com>
> Sent: Tuesday, November 1, 2022 2:01 AM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>
> Subject: RE: [PATCH 03/13] net/idpf: support device initialization
>
> > -----Original Message-----
> > From: Junfeng Guo <junfeng.guo@intel.com>
> > Sent: Wednesday, August 3, 2022 2:31 PM
> > To: qi.z.zhang@intel.com; jingjing.wu@intel.com; beilei.xing@intel.com
> > Cc: dev@dpdk.org; junfeng.guo@intel.com; Xiaoyun Li
> > <xiaoyun.li@intel.com>; Xiao Wang <xiao.w.wang@intel.com>
> > Subject: [PATCH 03/13] net/idpf: support device initialization
> >
> > Support device init and the following dev ops:
> > - dev_configure
> > - dev_start
> > - dev_stop
> > - dev_close
> >
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> > ---
>
> Hello,
>
> This patch is causing the following build failure in latest main (6a88cbc) with
> clang 3.4.2 in CentOS 7:
> drivers/net/idpf/idpf_vchnl.c:141:13: error: comparison of constant 522 with
> expression of type 'enum virtchnl_ops' is always false [-Werror,-
> Wtautological-constant-out-of-range-compare]
>
Hi,
Thanks for reporting the issue, fix patch has been sent, https://patches.dpdk.org/project/dpdk/patch/20221101024350.105241-1-beilei.xing@intel.com/
> Regards,
> Ali
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 03/13] net/idpf: support device initialization
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
2022-08-03 15:11 ` Stephen Hemminger
2022-10-31 18:00 ` Ali Alnubani
@ 2022-11-02 15:31 ` Raslan Darawsheh
2022-11-02 15:52 ` Thomas Monjalon
2022-11-03 0:56 ` Xing, Beilei
2 siblings, 2 replies; 376+ messages in thread
From: Raslan Darawsheh @ 2022-11-02 15:31 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: dev, Xiaoyun Li, Xiao Wang, NBU-Contact-Thomas Monjalon (EXTERNAL)
Hi,
> -----Original Message-----
> From: Junfeng Guo <junfeng.guo@intel.com>
> Sent: Wednesday, August 3, 2022 2:31 PM
> To: qi.z.zhang@intel.com; jingjing.wu@intel.com; beilei.xing@intel.com
> Cc: dev@dpdk.org; junfeng.guo@intel.com; Xiaoyun Li
> <xiaoyun.li@intel.com>; Xiao Wang <xiao.w.wang@intel.com>
> Subject: [PATCH 03/13] net/idpf: support device initialization
>
> Support device init and the following dev ops:
> - dev_configure
> - dev_start
> - dev_stop
> - dev_close
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
> ---
> drivers/net/idpf/idpf_ethdev.c | 651
> +++++++++++++++++++++++++++++++++ drivers/net/idpf/idpf_ethdev.h
> | 205 +++++++++++ drivers/net/idpf/idpf_vchnl.c | 476
> ++++++++++++++++++++++++
> drivers/net/idpf/meson.build | 18 +
> drivers/net/idpf/version.map | 3 +
> drivers/net/meson.build | 1 +
> 6 files changed, 1354 insertions(+)
> create mode 100644 drivers/net/idpf/idpf_ethdev.c create mode 100644
> drivers/net/idpf/idpf_ethdev.h create mode 100644
> drivers/net/idpf/idpf_vchnl.c create mode 100644
> drivers/net/idpf/meson.build create mode 100644
> drivers/net/idpf/version.map
>
> diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
> new file mode 100644 index 0000000000..87c68226dd
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.c
> @@ -0,0 +1,651 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +#include <rte_malloc.h>
> +#include <rte_memzone.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#define REPRESENTOR "representor"
> +
> +struct idpf_adapter *adapter;
> +uint16_t used_vecs_num;
> +
> +static const char * const idpf_valid_args[] = {
> + REPRESENTOR,
> + NULL
> +};
> +
> +static int idpf_dev_configure(struct rte_eth_dev *dev); static int
> +idpf_dev_start(struct rte_eth_dev *dev); static int
> +idpf_dev_stop(struct rte_eth_dev *dev); static int
> +idpf_dev_close(struct rte_eth_dev *dev);
> +
> +static const struct eth_dev_ops idpf_eth_dev_ops = {
> + .dev_configure = idpf_dev_configure,
> + .dev_start = idpf_dev_start,
> + .dev_stop = idpf_dev_stop,
> + .dev_close = idpf_dev_close,
> +};
> +
> +static int
> +idpf_init_vport_req_info(struct rte_eth_dev *dev) {
> + struct virtchnl2_create_vport *vport_info;
> + uint16_t idx = adapter->next_vport_idx;
> +
> + if (!adapter->vport_req_info[idx]) {
> + adapter->vport_req_info[idx] = rte_zmalloc(NULL,
> + sizeof(struct virtchnl2_create_vport), 0);
> + if (!adapter->vport_req_info[idx]) {
> + PMD_INIT_LOG(ERR, "Failed to allocate
> vport_req_info");
> + return -1;
> + }
> + }
> +
> + vport_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_req_info[idx];
> +
> + vport_info->vport_type =
> +rte_cpu_to_le_16(VIRTCHNL2_VPORT_TYPE_DEFAULT);
> +
> + return 0;
> +}
> +
> +static uint16_t
> +idpf_get_next_vport_idx(struct idpf_vport **vports, uint16_t
> max_vport_nb,
> + uint16_t cur_vport_idx)
> +{
> + uint16_t vport_idx;
> + uint16_t i;
> +
> + if (cur_vport_idx < max_vport_nb && !vports[cur_vport_idx + 1]) {
> + vport_idx = cur_vport_idx + 1;
> + return vport_idx;
> + }
> +
> + for (i = 0; i < max_vport_nb; i++) {
> + if (!vports[i])
> + break;
> + }
> +
> + if (i == max_vport_nb)
> + vport_idx = IDPF_INVALID_VPORT_IDX;
> + else
> + vport_idx = i;
> +
> + return vport_idx;
> +}
> +
> +#ifndef IDPF_RSS_KEY_LEN
> +#define IDPF_RSS_KEY_LEN 52
> +#endif
> +
> +static int
> +idpf_init_vport(struct rte_eth_dev *dev) {
> + uint16_t idx = adapter->next_vport_idx;
> + struct virtchnl2_create_vport *vport_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_recv_info[idx];
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> + int i;
> +
> + vport->adapter = adapter;
> + vport->vport_id = vport_info->vport_id;
> + vport->txq_model = vport_info->txq_model;
> + vport->rxq_model = vport_info->rxq_model;
> + vport->num_tx_q = vport_info->num_tx_q;
> + vport->num_tx_complq = vport_info->num_tx_complq;
> + vport->num_rx_q = vport_info->num_rx_q;
> + vport->num_rx_bufq = vport_info->num_rx_bufq;
> + vport->max_mtu = vport_info->max_mtu;
> + rte_memcpy(vport->default_mac_addr,
> + vport_info->default_mac_addr, ETH_ALEN);
> + vport->rss_algorithm = vport_info->rss_algorithm;
> + vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
> + vport_info->rss_key_size);
> + vport->rss_lut_size = vport_info->rss_lut_size;
> + vport->sw_idx = idx;
> +
> + for (i = 0; i < vport_info->chunks.num_chunks; i++) {
> + if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX) {
> + vport->chunks_info.tx_start_qid =
> + vport_info-
> >chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_qtail_start =
> + vport_info-
> >chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX) {
> + vport->chunks_info.rx_start_qid =
> + vport_info-
> >chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_qtail_start =
> + vport_info-
> >chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION) {
> + vport->chunks_info.tx_compl_start_qid =
> + vport_info-
> >chunks.chunks[i].start_queue_id;
> + vport->chunks_info.tx_compl_qtail_start =
> + vport_info-
> >chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.tx_compl_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + } else if (vport_info->chunks.chunks[i].type ==
> + VIRTCHNL2_QUEUE_TYPE_RX_BUFFER) {
> + vport->chunks_info.rx_buf_start_qid =
> + vport_info-
> >chunks.chunks[i].start_queue_id;
> + vport->chunks_info.rx_buf_qtail_start =
> + vport_info-
> >chunks.chunks[i].qtail_reg_start;
> + vport->chunks_info.rx_buf_qtail_spacing =
> + vport_info-
> >chunks.chunks[i].qtail_reg_spacing;
> + }
> + }
> +
> + adapter->vports[idx] = vport;
> + adapter->cur_vport_nb++;
> + adapter->next_vport_idx = idpf_get_next_vport_idx(adapter-
> >vports,
> + adapter->max_vport_nb,
> idx);
> + if (adapter->next_vport_idx == IDPF_INVALID_VPORT_IDX) {
> + PMD_INIT_LOG(ERR, "Failed to get next vport id");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_configure(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> + int ret = 0;
> +
> + ret = idpf_init_vport_req_info(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vport req_info.");
> + return ret;
> + }
> +
> + ret = idpf_create_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to create vport.");
> + return ret;
> + }
> +
> + ret = idpf_init_vport(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init vports.");
> + return ret;
> + }
> +
> + rte_ether_addr_copy((struct rte_ether_addr *)vport-
> >default_mac_addr,
> + &dev->data->mac_addrs[0]);
> +
> + return ret;
> +}
> +
> +static int
> +idpf_dev_start(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + vport->stopped = 0;
> +
> + if (idpf_ena_dis_vport(vport, true)) {
> + PMD_DRV_LOG(ERR, "Failed to enable vport");
> + goto err_vport;
> + }
> +
> + return 0;
> +
> +err_vport:
> + return -1;
> +}
> +
> +static int
> +idpf_dev_stop(struct rte_eth_dev *dev)
> +{
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + if (vport->stopped == 1)
> + return 0;
> +
> + if (idpf_ena_dis_vport(vport, false))
> + PMD_DRV_LOG(ERR, "disable vport failed");
> +
> + vport->stopped = 1;
> + dev->data->dev_started = 0;
> +
> + return 0;
> +}
> +
> +static int
> +idpf_dev_close(struct rte_eth_dev *dev) {
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return 0;
> +
> + idpf_dev_stop(dev);
> + idpf_destroy_vport(vport);
> +
> + return 0;
> +}
> +
> +static int idpf_parse_devargs(struct rte_eth_dev *dev) {
> + struct rte_devargs *devargs = dev->device->devargs;
> + struct rte_kvargs *kvlist;
> + int ret = 0;
> +
> + if (!devargs)
> + return 0;
> +
> + kvlist = rte_kvargs_parse(devargs->args, idpf_valid_args);
> + if (!kvlist) {
> + PMD_INIT_LOG(ERR, "invalid kvargs key");
> + return -EINVAL;
> + }
> +
> + rte_kvargs_free(kvlist);
> + return ret;
> +}
> +
> +static void
> +idpf_reset_pf(struct iecm_hw *hw)
> +{
> + uint32_t reg;
> +
> + reg = IECM_READ_REG(hw, PFGEN_CTRL);
> + IECM_WRITE_REG(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR)); }
> +
> +#define IDPF_RESET_WAIT_CNT 100
> +static int
> +idpf_check_pf_reset_done(struct iecm_hw *hw) {
> + uint32_t reg;
> + int i;
> +
> + for (i = 0; i < IDPF_RESET_WAIT_CNT; i++) {
> + reg = IECM_READ_REG(hw, PFGEN_RSTAT);
> + if (reg != 0xFFFFFFFF && (reg &
> PFGEN_RSTAT_PFR_STATE_M))
> + return 0;
> + rte_delay_ms(1000);
> + }
> +
> + PMD_INIT_LOG(ERR, "IDPF reset timeout");
> + return -EBUSY;
> +}
> +
> +#define CTLQ_NUM 2
> +static int
> +idpf_init_mbx(struct iecm_hw *hw)
> +{
> + struct iecm_ctlq_create_info ctlq_info[CTLQ_NUM] = {
> + {
> + .type = IECM_CTLQ_TYPE_MAILBOX_TX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ATQH,
> + .tail = PF_FW_ATQT,
> + .len = PF_FW_ATQLEN,
> + .bah = PF_FW_ATQBAH,
> + .bal = PF_FW_ATQBAL,
> + .len_mask = PF_FW_ATQLEN_ATQLEN_M,
> + .len_ena_mask =
> PF_FW_ATQLEN_ATQENABLE_M,
> + .head_mask = PF_FW_ATQH_ATQH_M,
> + }
> + },
> + {
> + .type = IECM_CTLQ_TYPE_MAILBOX_RX,
> + .id = IDPF_CTLQ_ID,
> + .len = IDPF_CTLQ_LEN,
> + .buf_size = IDPF_DFLT_MBX_BUF_SIZE,
> + .reg = {
> + .head = PF_FW_ARQH,
> + .tail = PF_FW_ARQT,
> + .len = PF_FW_ARQLEN,
> + .bah = PF_FW_ARQBAH,
> + .bal = PF_FW_ARQBAL,
> + .len_mask = PF_FW_ARQLEN_ARQLEN_M,
> + .len_ena_mask =
> PF_FW_ARQLEN_ARQENABLE_M,
> + .head_mask = PF_FW_ARQH_ARQH_M,
> + }
> + }
> + };
> + struct iecm_ctlq_info *ctlq;
> + int ret = 0;
> +
> + ret = iecm_ctlq_init(hw, CTLQ_NUM, ctlq_info);
> + if (ret)
> + return ret;
> +
> + LIST_FOR_EACH_ENTRY_SAFE(ctlq, NULL, &hw->cq_list_head,
> + struct iecm_ctlq_info, cq_list) {
> + if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type ==
> IECM_CTLQ_TYPE_MAILBOX_TX)
> + hw->asq = ctlq;
> + if (ctlq->q_id == IDPF_CTLQ_ID && ctlq->cq_type ==
> IECM_CTLQ_TYPE_MAILBOX_RX)
> + hw->arq = ctlq;
> + }
> +
> + if (!hw->asq || !hw->arq) {
> + iecm_ctlq_deinit(hw);
> + ret = -ENOENT;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +idpf_adapter_init(struct rte_eth_dev *dev) {
> + struct iecm_hw *hw = &adapter->hw;
> + struct rte_pci_device *pci_dev = IDPF_DEV_TO_PCI(dev);
> + int ret = 0;
> +
> + if (adapter->initialized)
> + return 0;
> +
> + hw->hw_addr = (void *)pci_dev->mem_resource[0].addr;
> + hw->hw_addr_len = pci_dev->mem_resource[0].len;
> + hw->back = adapter;
> + hw->vendor_id = pci_dev->id.vendor_id;
> + hw->device_id = pci_dev->id.device_id;
> + hw->subsystem_vendor_id = pci_dev->id.subsystem_vendor_id;
> +
> + ret = idpf_parse_devargs(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to parse devargs");
> + goto err;
> + }
> +
> + idpf_reset_pf(hw);
> + ret = idpf_check_pf_reset_done(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "IDPF is still resetting");
> + goto err;
> + }
> +
> + ret = idpf_init_mbx(hw);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init mailbox");
> + goto err;
> + }
> +
> + adapter->mbx_resp = rte_zmalloc("idpf_adapter_mbx_resp",
> IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->mbx_resp) {
> + PMD_INIT_LOG(ERR, "Failed to allocate
> idpf_adapter_mbx_resp memory");
> + goto err_mbx;
> + }
> +
> + if (idpf_check_api_version(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to check api version");
> + goto err_api;
> + }
> +
> + adapter->caps = rte_zmalloc("idpf_caps",
> + sizeof(struct virtchnl2_get_capabilities), 0);
> + if (!adapter->caps) {
> + PMD_INIT_LOG(ERR, "Failed to allocate idpf_caps memory");
> + goto err_api;
> + }
> +
> + if (idpf_get_caps(adapter)) {
> + PMD_INIT_LOG(ERR, "Failed to get capabilities");
> + goto err_caps;
> + }
> +
> + adapter->max_vport_nb = adapter->caps->max_vports;
> +
> + adapter->vport_req_info = rte_zmalloc("vport_req_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter-
> >vport_req_info),
> + 0);
> + if (!adapter->vport_req_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_req_info
> memory");
> + goto err_caps;
> + }
> +
> + adapter->vport_recv_info = rte_zmalloc("vport_recv_info",
> + adapter->max_vport_nb *
> + sizeof(*adapter-
> >vport_recv_info),
> + 0);
> + if (!adapter->vport_recv_info) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vport_recv_info
> memory");
> + goto err_vport_recv_info;
> + }
> +
> + adapter->vports = rte_zmalloc("vports",
> + adapter->max_vport_nb *
> + sizeof(*adapter->vports),
> + 0);
> + if (!adapter->vports) {
> + PMD_INIT_LOG(ERR, "Failed to allocate vports memory");
> + goto err_vports;
> + }
> +
> + adapter->max_rxq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_rx_queues)) /
> + sizeof(struct virtchnl2_rxq_info);
> + adapter->max_txq_per_msg = (IDPF_DFLT_MBX_BUF_SIZE -
> + sizeof(struct virtchnl2_config_tx_queues)) /
> + sizeof(struct virtchnl2_txq_info);
> +
> + adapter->cur_vport_nb = 0;
> + adapter->next_vport_idx = 0;
> + adapter->initialized = true;
> +
> + return ret;
> +
> +err_vports:
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> +err_vport_recv_info:
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> +err_caps:
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> +err_api:
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> +err_mbx:
> + iecm_ctlq_deinit(hw);
> +err:
> + return -1;
> +}
> +
> +
> +static int
> +idpf_dev_init(struct rte_eth_dev *dev, __rte_unused void *init_params)
> +{
> + struct idpf_vport *vport =
> + (struct idpf_vport *)dev->data->dev_private;
> + int ret = 0;
> +
> + PMD_INIT_FUNC_TRACE();
> +
> + dev->dev_ops = &idpf_eth_dev_ops;
> +
> + /* for secondary processes, we don't initialise any further as primary
> + * has already done this work.
> + */
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return ret;
> +
> + ret = idpf_adapter_init(dev);
> + if (ret) {
> + PMD_INIT_LOG(ERR, "Failed to init adapter.");
> + return ret;
> + }
> +
> + dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> +
> + vport->dev_data = dev->data;
> +
> + dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN,
> 0);
> + if (dev->data->mac_addrs == NULL) {
> + PMD_INIT_LOG(ERR, "Cannot allocate mac_addr memory.");
> + ret = -ENOMEM;
> + goto err;
> + }
> +
> +err:
> + return ret;
> +}
> +
> +static int
> +idpf_dev_uninit(struct rte_eth_dev *dev) {
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> + return -EPERM;
> +
> + idpf_dev_close(dev);
> +
> + return 0;
> +}
> +
> +static const struct rte_pci_id pci_id_idpf_map[] = {
> + { RTE_PCI_DEVICE(IECM_INTEL_VENDOR_ID, IECM_DEV_ID_PF) },
> + { .vendor_id = 0, /* sentinel */ },
> +};
> +
> +static int
> +idpf_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
> + struct rte_pci_device *pci_dev)
> +{
> + struct rte_eth_devargs eth_da = { .nb_representor_ports = 0 };
> + char name[RTE_ETH_NAME_MAX_LEN];
> + int i, retval;
> +
> + if (pci_dev->device.devargs) {
> + retval = rte_eth_devargs_parse(pci_dev->device.devargs-
> >args,
> + ð_da);
> + if (retval)
> + return retval;
> + }
> +
> + if (!eth_da.nb_representor_ports) {
> + PMD_INIT_LOG(ERR, "Failed to probe, need to add
> representor devargs.");
> + return -1;
> + }
> +
> + if (!adapter) {
> + adapter = (struct idpf_adapter *)rte_zmalloc("idpf_adapter",
> + sizeof(struct idpf_adapter), 0);
> + if (!adapter) {
> + PMD_INIT_LOG(ERR, "Failed to allocate adapter.");
> + return -1;
> + }
> + }
> +
> + for (i = 0; i < eth_da.nb_representor_ports; i++) {
> + snprintf(name, sizeof(name), "idpf_vport_%d",
> + eth_da.representor_ports[i]);
> + retval = rte_eth_dev_create(&pci_dev->device, name,
> + sizeof(struct idpf_vport),
> + NULL, NULL, idpf_dev_init,
> + NULL);
> + if (retval)
> + PMD_DRV_LOG(ERR, "failed to creat vport %d", i);
> + }
> +
> + return 0;
> +}
> +
> +static void
> +idpf_adapter_rel(struct idpf_adapter *adapter) {
> + struct iecm_hw *hw = &adapter->hw;
> + int i;
> +
> + iecm_ctlq_deinit(hw);
> +
> + if (adapter->caps) {
> + rte_free(adapter->caps);
> + adapter->caps = NULL;
> + }
> +
> + if (adapter->mbx_resp) {
> + rte_free(adapter->mbx_resp);
> + adapter->mbx_resp = NULL;
> + }
> +
> + if (adapter->vport_req_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + if (adapter->vport_req_info[i]) {
> + rte_free(adapter->vport_req_info[i]);
> + adapter->vport_req_info[i] = NULL;
> + }
> + }
> + rte_free(adapter->vport_req_info);
> + adapter->vport_req_info = NULL;
> + }
> +
> + if (adapter->vport_recv_info) {
> + for (i = 0; i < adapter->max_vport_nb; i++) {
> + if (adapter->vport_recv_info[i]) {
> + rte_free(adapter->vport_recv_info[i]);
> + adapter->vport_recv_info[i] = NULL;
> + }
> + }
> + }
> +
> + if (adapter->vports) {
> + /* Needn't free adapter->vports[i] since it's private data */
> + rte_free(adapter->vports);
> + adapter->vports = NULL;
> + }
> +}
> +
> +static int
> +idpf_pci_remove(struct rte_pci_device *pci_dev) {
> + if (adapter) {
> + idpf_adapter_rel(adapter);
> + rte_free(adapter);
> + adapter = NULL;
> + }
> +
> + return rte_eth_dev_pci_generic_remove(pci_dev, idpf_dev_uninit);
> }
> +
> +static struct rte_pci_driver rte_idpf_pmd = {
> + .id_table = pci_id_idpf_map,
> + .drv_flags = RTE_PCI_DRV_NEED_MAPPING |
> RTE_PCI_DRV_INTR_LSC |
> + RTE_PCI_DRV_PROBE_AGAIN,
> + .probe = idpf_pci_probe,
> + .remove = idpf_pci_remove,
> +};
> +
> +/**
> + * Driver initialization routine.
> + * Invoked once at EAL init time.
> + * Register itself as the [Poll Mode] Driver of PCI devices.
> + */
> +RTE_PMD_REGISTER_PCI(net_idpf, rte_idpf_pmd);
> +RTE_PMD_REGISTER_PCI_TABLE(net_idpf, pci_id_idpf_map);
> +RTE_PMD_REGISTER_KMOD_DEP(net_ice, "* igb_uio | uio_pci_generic |
> +vfio-pci");
> +
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_init, init, NOTICE);
> +RTE_LOG_REGISTER_SUFFIX(idpf_logtype_driver, driver, NOTICE);
> diff --git a/drivers/net/idpf/idpf_ethdev.h b/drivers/net/idpf/idpf_ethdev.h
> new file mode 100644 index 0000000000..501f772fa8
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_ethdev.h
> @@ -0,0 +1,205 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#ifndef _IDPF_ETHDEV_H_
> +#define _IDPF_ETHDEV_H_
> +
> +#include <stdint.h>
> +#include <rte_mbuf.h>
> +#include <rte_mempool.h>
> +#include <rte_malloc.h>
> +#include <rte_spinlock.h>
> +#include <rte_ethdev.h>
> +#include <rte_kvargs.h>
> +#include <ethdev_driver.h>
> +
> +#include "base/iecm_osdep.h"
> +#include "base/iecm_type.h"
> +#include "base/iecm_devids.h"
> +#include "base/iecm_lan_txrx.h"
> +#include "base/iecm_lan_pf_regs.h"
> +#include "base/virtchnl.h"
> +#include "base/virtchnl2.h"
> +
> +#define IDPF_INVALID_VPORT_IDX 0xffff
> +#define IDPF_TXQ_PER_GRP 1
> +#define IDPF_TX_COMPLQ_PER_GRP 1
> +#define IDPF_RXQ_PER_GRP 1
> +#define IDPF_RX_BUFQ_PER_GRP 2
> +
> +#define IDPF_CTLQ_ID -1
> +#define IDPF_CTLQ_LEN 64
> +#define IDPF_DFLT_MBX_BUF_SIZE 4096
> +
> +#define IDPF_DFLT_Q_VEC_NUM 1
> +#define IDPF_DFLT_INTERVAL 16
> +
> +#define IDPF_MAX_NUM_QUEUES 256
> +#define IDPF_MIN_BUF_SIZE 1024
> +#define IDPF_MAX_FRAME_SIZE 9728
> +
> +#define IDPF_NUM_MACADDR_MAX 64
> +
> +#define IDPF_MAX_PKT_TYPE 1024
> +
> +#define IDPF_VLAN_TAG_SIZE 4
> +#define IDPF_ETH_OVERHEAD \
> + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN +
> IDPF_VLAN_TAG_SIZE * 2)
> +
> +#ifndef ETH_ADDR_LEN
> +#define ETH_ADDR_LEN 6
> +#endif
> +
> +/* Message type read in virtual channel from PF */ enum idpf_vc_result
> +{
> + IDPF_MSG_ERR = -1, /* Meet error when accessing admin queue */
> + IDPF_MSG_NON, /* Read nothing from admin queue */
> + IDPF_MSG_SYS, /* Read system msg from admin queue */
> + IDPF_MSG_CMD, /* Read async command result */
> +};
> +
> +struct idpf_chunks_info {
> + uint32_t tx_start_qid;
> + uint32_t rx_start_qid;
> + /* Valid only if split queue model */
> + uint32_t tx_compl_start_qid;
> + uint32_t rx_buf_start_qid;
> +
> + uint64_t tx_qtail_start;
> + uint32_t tx_qtail_spacing;
> + uint64_t rx_qtail_start;
> + uint32_t rx_qtail_spacing;
> + uint64_t tx_compl_qtail_start;
> + uint32_t tx_compl_qtail_spacing;
> + uint64_t rx_buf_qtail_start;
> + uint32_t rx_buf_qtail_spacing;
> +};
> +
> +struct idpf_vport {
> + struct idpf_adapter *adapter; /* Backreference to associated
> adapter */
> + uint16_t vport_id;
> + uint32_t txq_model;
> + uint32_t rxq_model;
> + uint16_t num_tx_q;
> + /* valid only if txq_model is split Q */
> + uint16_t num_tx_complq;
> + uint16_t num_rx_q;
> + /* valid only if rxq_model is split Q */
> + uint16_t num_rx_bufq;
> +
> + uint16_t max_mtu;
> + uint8_t default_mac_addr[VIRTCHNL_ETH_LENGTH_OF_ADDRESS];
> +
> + enum virtchnl_rss_algorithm rss_algorithm;
> + uint16_t rss_key_size;
> + uint16_t rss_lut_size;
> +
> + uint16_t sw_idx; /* SW idx */
> +
> + struct rte_eth_dev_data *dev_data; /* Pointer to the device data */
> + uint16_t max_pkt_len; /* Maximum packet length */
> +
> + /* RSS info */
> + uint32_t *rss_lut;
> + uint8_t *rss_key;
> + uint64_t rss_hf;
> +
> + /* Chunk info */
> + struct idpf_chunks_info chunks_info;
> +
> + /* Event from ipf */
> + bool link_up;
> + uint32_t link_speed;
> +
> + bool stopped;
> + struct virtchnl2_vport_stats eth_stats_offset; };
> +
> +struct idpf_adapter {
> + struct iecm_hw hw;
> +
> + struct virtchnl_version_info virtchnl_version;
> + struct virtchnl2_get_capabilities *caps;
> +
> + volatile enum virtchnl_ops pend_cmd; /* pending command not
> finished */
> + uint32_t cmd_retval; /* return value of the cmd response from ipf */
> + uint8_t *mbx_resp; /* buffer to store the mailbox response from ipf
> */
> +
> + uint32_t txq_model;
> + uint32_t rxq_model;
> +
> + /* Vport info */
> + uint8_t **vport_req_info;
> + uint8_t **vport_recv_info;
> + struct idpf_vport **vports;
> + uint16_t max_vport_nb;
> + uint16_t cur_vport_nb;
> + uint16_t next_vport_idx;
> +
> + /* Max config queue number per VC message */
> + uint32_t max_rxq_per_msg;
> + uint32_t max_txq_per_msg;
> +
> + uint32_t ptype_tbl[IDPF_MAX_PKT_TYPE]
> __rte_cache_min_aligned;
> +
> + bool initialized;
> + bool stopped;
> +};
> +
> +extern struct idpf_adapter *adapter;
> +
> +#define IDPF_DEV_TO_PCI(eth_dev) \
> + RTE_DEV_TO_PCI((eth_dev)->device)
> +
> +/* structure used for sending and checking response of virtchnl ops */
> +struct idpf_cmd_info {
> + uint32_t ops;
> + uint8_t *in_args; /* buffer for sending */
> + uint32_t in_args_size; /* buffer size for sending */
> + uint8_t *out_buffer; /* buffer for response */
> + uint32_t out_size; /* buffer size for response */
> +};
> +
> +/* notify current command done. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_notify_cmd(struct idpf_adapter *adapter, int msg_ret) {
> + adapter->cmd_retval = msg_ret;
> + rte_wmb();
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN; }
> +
> +/* clear current command. Only call in case execute
> + * _atomic_set_cmd successfully.
> + */
> +static inline void
> +_clear_cmd(struct idpf_adapter *adapter) {
> + rte_wmb();
> + adapter->pend_cmd = VIRTCHNL_OP_UNKNOWN;
> + adapter->cmd_retval = VIRTCHNL_STATUS_SUCCESS; }
> +
> +/* Check there is pending cmd in execution. If none, set new command.
> +*/ static inline int _atomic_set_cmd(struct idpf_adapter *adapter, enum
> +virtchnl_ops ops) {
> + int ret = rte_atomic32_cmpset(&adapter->pend_cmd,
> VIRTCHNL_OP_UNKNOWN,
> +ops);
> +
> + if (!ret)
> + PMD_DRV_LOG(ERR, "There is incomplete cmd %d",
> adapter->pend_cmd);
> +
> + return !ret;
> +}
> +
> +void idpf_handle_virtchnl_msg(struct rte_eth_dev *dev); int
> +idpf_check_api_version(struct idpf_adapter *adapter); int
> +idpf_get_caps(struct idpf_adapter *adapter); int
> +idpf_create_vport(__rte_unused struct rte_eth_dev *dev); int
> +idpf_destroy_vport(struct idpf_vport *vport); int
> +idpf_ena_dis_vport(struct idpf_vport *vport, bool enable);
> +
> +#endif /* _IDPF_ETHDEV_H_ */
> diff --git a/drivers/net/idpf/idpf_vchnl.c b/drivers/net/idpf/idpf_vchnl.c new
> file mode 100644 index 0000000000..4fc15d5b71
> --- /dev/null
> +++ b/drivers/net/idpf/idpf_vchnl.c
> @@ -0,0 +1,476 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 Intel Corporation
> + */
> +
> +#include <stdio.h>
> +#include <errno.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <stdarg.h>
> +#include <inttypes.h>
> +#include <rte_byteorder.h>
> +#include <rte_common.h>
> +
> +#include <rte_debug.h>
> +#include <rte_atomic.h>
> +#include <rte_eal.h>
> +#include <rte_ether.h>
> +#include <ethdev_driver.h>
> +#include <ethdev_pci.h>
> +#include <rte_dev.h>
> +
> +#include "idpf_ethdev.h"
> +
> +#include "base/iecm_prototype.h"
> +
> +#define IDPF_CTLQ_LEN 64
> +
> +static int
> +idpf_vc_clean(struct idpf_adapter *adapter) {
> + struct iecm_ctlq_msg *q_msg[IDPF_CTLQ_LEN];
> + uint16_t num_q_msg = IDPF_CTLQ_LEN;
> + struct iecm_dma_mem *dma_mem;
> + int err = 0;
> + uint32_t i;
> +
> + for (i = 0; i < 10; i++) {
> + err = iecm_ctlq_clean_sq(adapter->hw.asq, &num_q_msg,
> q_msg);
> + msleep(20);
> + if (num_q_msg)
> + break;
> + }
> + if (err)
> + goto error;
> +
> + /* Empty queue is not an error */
> + for (i = 0; i < num_q_msg; i++) {
> + dma_mem = q_msg[i]->ctx.indirect.payload;
> + if (dma_mem) {
> + iecm_free_dma_mem(&adapter->hw, dma_mem);
> + rte_free(dma_mem);
> + }
> + rte_free(q_msg[i]);
> + }
> +
> +error:
> + return err;
> +}
> +
> +static int
> +idpf_send_vc_msg(struct idpf_adapter *adapter, enum virtchnl_ops op,
> + uint16_t msg_size, uint8_t *msg)
> +{
> + struct iecm_ctlq_msg *ctlq_msg;
> + struct iecm_dma_mem *dma_mem;
> + int err = 0;
> +
> + err = idpf_vc_clean(adapter);
> + if (err)
> + goto err;
> +
> + ctlq_msg = (struct iecm_ctlq_msg *)rte_zmalloc(NULL,
> + sizeof(struct iecm_ctlq_msg), 0);
> + if (!ctlq_msg) {
> + err = -ENOMEM;
> + goto err;
> + }
> +
> + dma_mem = (struct iecm_dma_mem *)rte_zmalloc(NULL,
> + sizeof(struct iecm_dma_mem), 0);
> + if (!dma_mem) {
> + err = -ENOMEM;
> + goto dma_mem_error;
> + }
> +
> + dma_mem->size = IDPF_DFLT_MBX_BUF_SIZE;
> + iecm_alloc_dma_mem(&adapter->hw, dma_mem, dma_mem-
> >size);
> + if (!dma_mem->va) {
> + err = -ENOMEM;
> + goto dma_alloc_error;
> + }
> +
> + memcpy(dma_mem->va, msg, msg_size);
> +
> + ctlq_msg->opcode = iecm_mbq_opc_send_msg_to_pf;
> + ctlq_msg->func_id = 0;
> + ctlq_msg->data_len = msg_size;
> + ctlq_msg->cookie.mbx.chnl_opcode = op;
> + ctlq_msg->cookie.mbx.chnl_retval = VIRTCHNL_STATUS_SUCCESS;
> + ctlq_msg->ctx.indirect.payload = dma_mem;
> +
> + err = iecm_ctlq_send(&adapter->hw, adapter->hw.asq, 1, ctlq_msg);
> + if (err)
> + goto send_error;
> +
> + return err;
> +
> +send_error:
> + iecm_free_dma_mem(&adapter->hw, dma_mem);
> +dma_alloc_error:
> + rte_free(dma_mem);
> +dma_mem_error:
> + rte_free(ctlq_msg);
> +err:
> + return err;
> +}
> +
> +static enum idpf_vc_result
> +idpf_read_msg_from_ipf(struct idpf_adapter *adapter, uint16_t buf_len,
> + uint8_t *buf)
> +{
> + struct iecm_hw *hw = &adapter->hw;
> + struct iecm_ctlq_msg ctlq_msg;
> + struct iecm_dma_mem *dma_mem = NULL;
> + enum idpf_vc_result result = IDPF_MSG_NON;
> + enum virtchnl_ops opcode;
> + uint16_t pending = 1;
> + int ret;
> +
> + ret = iecm_ctlq_recv(hw->arq, &pending, &ctlq_msg);
> + if (ret) {
> + PMD_DRV_LOG(DEBUG, "Can't read msg from AQ");
> + if (ret != IECM_ERR_CTLQ_NO_WORK)
> + result = IDPF_MSG_ERR;
> + return result;
> + }
> +
> + rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
> +
> + opcode = (enum
> virtchnl_ops)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
> + adapter->cmd_retval =
> + (enum
> +virtchnl_status_code)rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
> +
> + PMD_DRV_LOG(DEBUG, "CQ from ipf carries opcode %u, retval %d",
> + opcode, adapter->cmd_retval);
> +
> + if (opcode == VIRTCHNL2_OP_EVENT) {
> + struct virtchnl2_event *ve =
> + (struct virtchnl2_event
> *)ctlq_msg.ctx.indirect.payload->va;
> +
> + result = IDPF_MSG_SYS;
> + switch (ve->event) {
> + case VIRTCHNL2_EVENT_LINK_CHANGE:
> + /* TBD */
> + break;
> + default:
> + PMD_DRV_LOG(ERR, "%s: Unknown event %d from
> ipf",
> + __func__, ve->event);
> + break;
> + }
> + } else {
> + /* async reply msg on command issued by pf previously */
> + result = IDPF_MSG_CMD;
> + if (opcode != adapter->pend_cmd) {
> + PMD_DRV_LOG(WARNING, "command mismatch,
> expect %u, get %u",
> + adapter->pend_cmd, opcode);
> + result = IDPF_MSG_ERR;
> + }
> + }
> +
> + if (ctlq_msg.data_len)
> + dma_mem = ctlq_msg.ctx.indirect.payload;
> + else
> + pending = 0;
> +
> + ret = iecm_ctlq_post_rx_buffs(hw, hw->arq, &pending,
> &dma_mem);
> + if (ret && dma_mem)
> + iecm_free_dma_mem(hw, dma_mem);
> +
> + return result;
> +}
> +
> +#define MAX_TRY_TIMES 200
> +#define ASQ_DELAY_MS 10
> +
> +static int
> +idpf_execute_vc_cmd(struct idpf_adapter *adapter, struct idpf_cmd_info
> +*args) {
> + enum idpf_vc_result result;
> + int err = 0;
> + int i = 0;
> + int ret;
> +
> + if (_atomic_set_cmd(adapter, args->ops))
> + return -1;
> +
> + ret = idpf_send_vc_msg(adapter, args->ops,
> + args->in_args_size,
> + args->in_args);
> + if (ret) {
> + PMD_DRV_LOG(ERR, "fail to send cmd %d", args->ops);
> + _clear_cmd(adapter);
> + return ret;
> + }
> +
> + switch (args->ops) {
> + case VIRTCHNL_OP_VERSION:
> + case VIRTCHNL2_OP_GET_CAPS:
> + case VIRTCHNL2_OP_CREATE_VPORT:
> + case VIRTCHNL2_OP_DESTROY_VPORT:
> + case VIRTCHNL2_OP_SET_RSS_KEY:
> + case VIRTCHNL2_OP_SET_RSS_LUT:
> + case VIRTCHNL2_OP_SET_RSS_HASH:
> + case VIRTCHNL2_OP_CONFIG_RX_QUEUES:
> + case VIRTCHNL2_OP_CONFIG_TX_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_QUEUES:
> + case VIRTCHNL2_OP_DISABLE_QUEUES:
> + case VIRTCHNL2_OP_ENABLE_VPORT:
> + case VIRTCHNL2_OP_DISABLE_VPORT:
> + /* for init virtchnl ops, need to poll the response */
> + do {
> + result = idpf_read_msg_from_ipf(adapter,
> + args->out_size,
> + args->out_buffer);
> + if (result == IDPF_MSG_CMD)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + } while (i++ < MAX_TRY_TIMES);
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure
> (%d) for cmd %d",
> + adapter->cmd_retval, args->ops);
> + }
> + _clear_cmd(adapter);
> + break;
> + default:
> + /* For other virtchnl ops in running time,
> + * wait for the cmd done flag.
> + */
> + do {
> + if (adapter->pend_cmd ==
> VIRTCHNL_OP_UNKNOWN)
> + break;
> + rte_delay_ms(ASQ_DELAY_MS);
> + /* If don't read msg or read sys event, continue */
> + } while (i++ < MAX_TRY_TIMES);
> + /* If there's no response is received, clear command */
> + if (i >= MAX_TRY_TIMES ||
> + adapter->cmd_retval != VIRTCHNL_STATUS_SUCCESS) {
> + err = -1;
> + PMD_DRV_LOG(ERR, "No response or return failure
> (%d) for cmd %d",
> + adapter->cmd_retval, args->ops);
> + _clear_cmd(adapter);
> + }
> + break;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_check_api_version(struct idpf_adapter *adapter) {
> + struct virtchnl_version_info version;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&version, 0, sizeof(struct virtchnl_version_info));
> + version.major = VIRTCHNL_VERSION_MAJOR_2;
> + version.minor = VIRTCHNL_VERSION_MINOR_0;
> +
> + args.ops = VIRTCHNL_OP_VERSION;
> + args.in_args = (uint8_t *)&version;
> + args.in_args_size = sizeof(version);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL_OP_VERSION");
> + return err;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_get_caps(struct idpf_adapter *adapter) {
> + struct virtchnl2_get_capabilities caps_msg;
> + struct idpf_cmd_info args;
> + int err;
> +
> + memset(&caps_msg, 0, sizeof(struct virtchnl2_get_capabilities));
> + caps_msg.csum_caps =
> + VIRTCHNL2_CAP_TX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_TX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_TX_CSUM_GENERIC |
> + VIRTCHNL2_CAP_RX_CSUM_L3_IPV4 |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV4_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_TCP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_UDP |
> + VIRTCHNL2_CAP_RX_CSUM_L4_IPV6_SCTP |
> + VIRTCHNL2_CAP_RX_CSUM_GENERIC;
> +
> + caps_msg.seg_caps =
> + VIRTCHNL2_CAP_SEG_IPV4_TCP |
> + VIRTCHNL2_CAP_SEG_IPV4_UDP |
> + VIRTCHNL2_CAP_SEG_IPV4_SCTP |
> + VIRTCHNL2_CAP_SEG_IPV6_TCP |
> + VIRTCHNL2_CAP_SEG_IPV6_UDP |
> + VIRTCHNL2_CAP_SEG_IPV6_SCTP |
> + VIRTCHNL2_CAP_SEG_GENERIC;
> +
> + caps_msg.rss_caps =
> + VIRTCHNL2_CAP_RSS_IPV4_TCP |
> + VIRTCHNL2_CAP_RSS_IPV4_UDP |
> + VIRTCHNL2_CAP_RSS_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV4_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV6_TCP |
> + VIRTCHNL2_CAP_RSS_IPV6_UDP |
> + VIRTCHNL2_CAP_RSS_IPV6_SCTP |
> + VIRTCHNL2_CAP_RSS_IPV6_OTHER |
> + VIRTCHNL2_CAP_RSS_IPV4_AH |
> + VIRTCHNL2_CAP_RSS_IPV4_ESP |
> + VIRTCHNL2_CAP_RSS_IPV4_AH_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH |
> + VIRTCHNL2_CAP_RSS_IPV6_ESP |
> + VIRTCHNL2_CAP_RSS_IPV6_AH_ESP;
> +
> + caps_msg.hsplit_caps =
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L2 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L3 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V4 |
> + VIRTCHNL2_CAP_RX_HSPLIT_AT_L4V6;
> +
> + caps_msg.rsc_caps =
> + VIRTCHNL2_CAP_RSC_IPV4_TCP |
> + VIRTCHNL2_CAP_RSC_IPV4_SCTP |
> + VIRTCHNL2_CAP_RSC_IPV6_TCP |
> + VIRTCHNL2_CAP_RSC_IPV6_SCTP;
> +
> + caps_msg.other_caps =
> + VIRTCHNL2_CAP_RDMA |
> + VIRTCHNL2_CAP_SRIOV |
> + VIRTCHNL2_CAP_MACFILTER |
> + VIRTCHNL2_CAP_FLOW_DIRECTOR |
> + VIRTCHNL2_CAP_SPLITQ_QSCHED |
> + VIRTCHNL2_CAP_CRC |
> + VIRTCHNL2_CAP_WB_ON_ITR |
> + VIRTCHNL2_CAP_PROMISC |
> + VIRTCHNL2_CAP_LINK_SPEED |
> + VIRTCHNL2_CAP_VLAN;
> +
> + args.ops = VIRTCHNL2_OP_GET_CAPS;
> + args.in_args = (uint8_t *)&caps_msg;
> + args.in_args_size = sizeof(caps_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL2_OP_GET_CAPS");
> + return err;
> + }
> +
> + rte_memcpy(adapter->caps, args.out_buffer, sizeof(caps_msg));
> +
> + return err;
> +}
> +
> +int
> +idpf_create_vport(__rte_unused struct rte_eth_dev *dev) {
> + uint16_t idx = adapter->next_vport_idx;
> + struct virtchnl2_create_vport *vport_req_info =
> + (struct virtchnl2_create_vport *)adapter-
> >vport_req_info[idx];
> + struct virtchnl2_create_vport vport_msg;
> + struct idpf_cmd_info args;
> + int err = -1;
> +
> + memset(&vport_msg, 0, sizeof(struct virtchnl2_create_vport));
> + vport_msg.vport_type = vport_req_info->vport_type;
> + vport_msg.txq_model = vport_req_info->txq_model;
> + vport_msg.rxq_model = vport_req_info->rxq_model;
> + vport_msg.num_tx_q = vport_req_info->num_tx_q;
> + vport_msg.num_tx_complq = vport_req_info->num_tx_complq;
> + vport_msg.num_rx_q = vport_req_info->num_rx_q;
> + vport_msg.num_rx_bufq = vport_req_info->num_rx_bufq;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_CREATE_VPORT;
> + args.in_args = (uint8_t *)&vport_msg;
> + args.in_args_size = sizeof(vport_msg);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR,
> + "Failed to execute command of
> VIRTCHNL2_OP_CREATE_VPORT");
> + return err;
> + }
> +
> + if (!adapter->vport_recv_info[idx]) {
> + adapter->vport_recv_info[idx] = rte_zmalloc(NULL,
> +
> IDPF_DFLT_MBX_BUF_SIZE, 0);
> + if (!adapter->vport_recv_info[idx]) {
> + PMD_INIT_LOG(ERR, "Failed to alloc
> vport_recv_info.");
> + return err;
> + }
> + }
> + rte_memcpy(adapter->vport_recv_info[idx], args.out_buffer,
> + IDPF_DFLT_MBX_BUF_SIZE);
> + return err;
> +}
> +
> +int
> +idpf_destroy_vport(struct idpf_vport *vport) {
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> +
> + memset(&args, 0, sizeof(args));
> + args.ops = VIRTCHNL2_OP_DESTROY_VPORT;
> + args.in_args = (uint8_t *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of
> VIRTCHNL2_OP_DESTROY_VPORT");
> + return err;
> + }
> +
> + return err;
> +}
> +
> +int
> +idpf_ena_dis_vport(struct idpf_vport *vport, bool enable) {
> + struct virtchnl2_vport vc_vport;
> + struct idpf_cmd_info args;
> + int err;
> +
> + vc_vport.vport_id = vport->vport_id;
> + args.ops = enable ? VIRTCHNL2_OP_ENABLE_VPORT :
> + VIRTCHNL2_OP_DISABLE_VPORT;
> + args.in_args = (u8 *)&vc_vport;
> + args.in_args_size = sizeof(vc_vport);
> + args.out_buffer = adapter->mbx_resp;
> + args.out_size = IDPF_DFLT_MBX_BUF_SIZE;
> +
> + err = idpf_execute_vc_cmd(adapter, &args);
> + if (err) {
> + PMD_DRV_LOG(ERR, "Failed to execute command of
> VIRTCHNL2_OP_%s_VPORT",
> + enable ? "ENABLE" : "DISABLE");
> + }
> +
> + return err;
> +}
> +
> diff --git a/drivers/net/idpf/meson.build b/drivers/net/idpf/meson.build
> new file mode 100644 index 0000000000..3a84162f93
> --- /dev/null
> +++ b/drivers/net/idpf/meson.build
> @@ -0,0 +1,18 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022 Intel
> +Corporation
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
> +endif
> +
> +subdir('base')
> +objs = [base_objs]
> +
> +sources = files(
> + 'idpf_ethdev.c',
> + 'idpf_vchnl.c',
> +)
> +
> +includes += include_directories('base')
> diff --git a/drivers/net/idpf/version.map b/drivers/net/idpf/version.map
> new file mode 100644 index 0000000000..b7da224860
> --- /dev/null
> +++ b/drivers/net/idpf/version.map
> @@ -0,0 +1,3 @@
> +DPDK_22 {
> + local: *;
> +};
> \ No newline at end of file
> diff --git a/drivers/net/meson.build b/drivers/net/meson.build index
> e35652fe63..8910154544 100644
> --- a/drivers/net/meson.build
> +++ b/drivers/net/meson.build
> @@ -28,6 +28,7 @@ drivers = [
> 'i40e',
> 'iavf',
> 'ice',
> + 'idpf',
> 'igc',
> 'ionic',
> 'ipn3ke',
> --
> 2.25.1
I'd like to report that this patch is causing a compilation failure over the main tree 22.11-rc2:
this is the failure which I see:
ing-field-initializers -D_GNU_SOURCE -fPIC -march=native -mno-avx512f -mno-avx512f -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -DRTE_LOG_DEFAULT_LOGTYPE=pmd.net.idpf -MD -MQ drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -MF drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o.d -o drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -c ../../root/dpdk/drivers/net/idpf/idpf_vchnl.c
../../root/dpdk/drivers/net/idpf/idpf_vchnl.c:142:13: error: comparison of constant 522 with expression of type 'enum virtchnl_ops' is always false [-Werror,-Wtautological-constant-out-of-range-compare]
if (opcode == VIRTCHNL2_OP_EVENT) {
~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
1 error generated.
[1424/2559] Generating eal.sym_chk with a meson_exe.py custom command
ninja: build stopped: subcommand failed.
And it's happening with the CLANG compiler:
clang version 3.4.2 (tags/RELEASE_34/dot2-final)
Kindest regards,
Raslan Darawsheh
^ permalink raw reply [flat|nested] 376+ messages in thread
* Re: [PATCH 03/13] net/idpf: support device initialization
2022-11-02 15:31 ` Raslan Darawsheh
@ 2022-11-02 15:52 ` Thomas Monjalon
2022-11-03 0:56 ` Xing, Beilei
1 sibling, 0 replies; 376+ messages in thread
From: Thomas Monjalon @ 2022-11-02 15:52 UTC (permalink / raw)
To: Junfeng Guo, qi.z.zhang, jingjing.wu, beilei.xing
Cc: Raslan Darawsheh, dev, Xiaoyun Li, Xiao Wang, david.marchand
02/11/2022 16:31, Raslan Darawsheh:
> > Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> > Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> > Signed-off-by: Junfeng Guo <junfeng.guo@intel.com>
[...]
> I'd like to report that this patch is causing a compilation failure over the main tree 22.11-rc2:
>
> this is the failure which I see:
> ing-field-initializers -D_GNU_SOURCE -fPIC -march=native -mno-avx512f -mno-avx512f -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -DRTE_LOG_DEFAULT_LOGTYPE=pmd.net.idpf -MD -MQ drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -MF drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o.d -o drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -c ../../root/dpdk/drivers/net/idpf/idpf_vchnl.c
> ../../root/dpdk/drivers/net/idpf/idpf_vchnl.c:142:13: error: comparison of constant 522 with expression of type 'enum virtchnl_ops' is always false [-Werror,-Wtautological-constant-out-of-range-compare]
> if (opcode == VIRTCHNL2_OP_EVENT) {
> ~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
> 1 error generated.
> [1424/2559] Generating eal.sym_chk with a meson_exe.py custom command
> ninja: build stopped: subcommand failed.
>
> And it's happening with the CLANG compiler:
> clang version 3.4.2 (tags/RELEASE_34/dot2-final)
That's unfortunate.
Please Beilei, Xiaoyun, Xiao and Junfeng,
is it something you could fix quickly?
Thanks
^ permalink raw reply [flat|nested] 376+ messages in thread
* RE: [PATCH 03/13] net/idpf: support device initialization
2022-11-02 15:31 ` Raslan Darawsheh
2022-11-02 15:52 ` Thomas Monjalon
@ 2022-11-03 0:56 ` Xing, Beilei
1 sibling, 0 replies; 376+ messages in thread
From: Xing, Beilei @ 2022-11-03 0:56 UTC (permalink / raw)
To: Raslan Darawsheh, Guo, Junfeng, Zhang, Qi Z, Wu, Jingjing
Cc: dev, Li, Xiaoyun, Wang, Xiao W, NBU-Contact-Thomas Monjalon (EXTERNAL)
> -----Original Message-----
> From: Raslan Darawsheh <rasland@nvidia.com>
> Sent: Wednesday, November 2, 2022 11:31 PM
> To: Guo, Junfeng <junfeng.guo@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Li, Xiaoyun <xiaoyun.li@intel.com>; Wang, Xiao W
> <xiao.w.wang@intel.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>
> Subject: RE: [PATCH 03/13] net/idpf: support device initialization
>
> Hi,
>
[snip]
> > --
> > 2.25.1
> I'd like to report that this patch is causing a compilation failure over the main
> tree 22.11-rc2:
>
> this is the failure which I see:
> ing-field-initializers -D_GNU_SOURCE -fPIC -march=native -mno-avx512f -
> mno-avx512f -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -
> DRTE_LOG_DEFAULT_LOGTYPE=pmd.net.idpf -MD -MQ
> drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -MF
> drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o.d -o
> drivers/libtmp_rte_net_idpf.a.p/net_idpf_idpf_vchnl.c.o -
> c ../../root/dpdk/drivers/net/idpf/idpf_vchnl.c
> ../../root/dpdk/drivers/net/idpf/idpf_vchnl.c:142:13: error: comparison of
> constant 522 with expression of type 'enum virtchnl_ops' is always false [-
> Werror,-Wtautological-constant-out-of-range-compare]
> if (opcode == VIRTCHNL2_OP_EVENT) {
> ~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
> 1 error generated.
> [1424/2559] Generating eal.sym_chk with a meson_exe.py custom command
> ninja: build stopped: subcommand failed.
>
> And it's happening with the CLANG compiler:
> clang version 3.4.2 (tags/RELEASE_34/dot2-final)
>
> Kindest regards,
> Raslan Darawsheh
Thanks for reporting, it should be fixed with the fix patch https://patches.dpdk.org/project/dpdk/patch/20221101024350.105241-1-beilei.xing@intel.com/.
BR,
Beilei
^ permalink raw reply [flat|nested] 376+ messages in thread
end of thread, other threads:[~2022-11-03 0:57 UTC | newest]
Thread overview: 376+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-03 11:30 [PATCH 00/13] add support for idpf PMD in DPDK Junfeng Guo
2022-08-03 11:30 ` [PATCH 01/13] net/idpf/base: introduce base code Junfeng Guo
2022-08-03 11:30 ` [PATCH 02/13] net/idpf/base: add logs and OS specific implementation Junfeng Guo
2022-08-03 11:30 ` [PATCH 03/13] net/idpf: support device initialization Junfeng Guo
2022-08-03 15:11 ` Stephen Hemminger
2022-08-08 4:43 ` Guo, Junfeng
2022-10-31 18:00 ` Ali Alnubani
2022-11-01 6:55 ` Xing, Beilei
2022-11-02 15:31 ` Raslan Darawsheh
2022-11-02 15:52 ` Thomas Monjalon
2022-11-03 0:56 ` Xing, Beilei
2022-08-03 11:30 ` [PATCH 04/13] net/idpf: add queue operations Junfeng Guo
2022-08-03 15:16 ` Stephen Hemminger
2022-08-08 4:44 ` Guo, Junfeng
2022-08-03 11:30 ` [PATCH 05/13] net/idpf: add support to get device information Junfeng Guo
2022-08-03 11:30 ` [PATCH 06/13] net/idpf: add support to get packet type Junfeng Guo
2022-08-03 11:30 ` [PATCH 07/13] net/idpf: add support to update link status Junfeng Guo
2022-08-03 11:30 ` [PATCH 08/13] net/idpf: add basic Rx/Tx datapath Junfeng Guo
2022-08-03 11:31 ` [PATCH 09/13] net/idpf: add support for RSS Junfeng Guo
2022-08-03 11:31 ` [PATCH 10/13] net/idpf: add mtu configuration Junfeng Guo
2022-08-03 11:31 ` [PATCH 11/13] net/idpf: add hw statistics Junfeng Guo
2022-08-03 11:31 ` [PATCH 12/13] net/idpf: support write back based on ITR expire Junfeng Guo
2022-08-03 11:31 ` [PATCH 13/13] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 01/14] net/idpf/base: introduce base code Junfeng Guo
2022-10-03 13:20 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-09-05 10:58 ` [PATCH v2 02/14] net/idpf/base: add logs and OS specific implementation Junfeng Guo
2022-10-03 13:20 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-10-12 8:07 ` Wu, Wenjun1
2022-09-05 10:58 ` [PATCH v2 03/14] net/idpf: add support for device initialization Junfeng Guo
2022-09-21 5:41 ` Xing, Beilei
2022-09-21 6:04 ` Xing, Beilei
2022-10-03 13:44 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-10-10 7:48 ` Wu, Wenjun1
2022-09-05 10:58 ` [PATCH v2 04/14] net/idpf: add support for queue operations Junfeng Guo
2022-10-03 13:47 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-09-05 10:58 ` [PATCH v2 05/14] net/idpf: add support for device information get Junfeng Guo
2022-10-03 13:53 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 06/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-03 13:58 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-09-05 10:58 ` [PATCH v2 07/14] net/idpf: add support for link status update Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-03 14:02 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-09-05 10:58 ` [PATCH v2 09/14] net/idpf: add support for RSS Junfeng Guo
2022-10-03 14:10 ` Andrew Rybchenko
2022-09-05 10:58 ` [PATCH v2 10/14] net/idpf: add support for mtu configuration Junfeng Guo
2022-10-03 14:12 ` Andrew Rybchenko
2022-10-14 9:18 ` Guo, Junfeng
2022-09-05 10:58 ` [PATCH v2 11/14] net/idpf: add support for hw statistics Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-09-05 10:58 ` [PATCH v2 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-03 14:20 ` Andrew Rybchenko
2022-10-14 9:19 ` Guo, Junfeng
2022-10-10 8:06 ` Wu, Wenjun1
2022-09-05 10:58 ` [PATCH v2 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-03 14:22 ` Andrew Rybchenko
2022-10-14 9:19 ` Guo, Junfeng
2022-10-10 7:56 ` Wu, Wenjun1
2022-10-03 13:31 ` [PATCH v2 00/14] add support for idpf PMD in DPDK Andrew Rybchenko
2022-10-03 14:36 ` Andrew Rybchenko
2022-10-18 11:09 ` Guo, Junfeng
2022-10-18 11:12 ` [PATCH v3 00/15] " Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 01/15] common/idpf: introduce common library Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-20 2:41 ` [PATCH v7 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 01/14] common/idpf: introduce common library Junfeng Guo
2022-10-21 6:40 ` Andrew Rybchenko
2022-10-21 12:35 ` Xing, Beilei
2022-10-21 12:38 ` Andrew Rybchenko
2022-10-21 12:46 ` Zhang, Qi Z
2022-10-24 13:01 ` [PATCH v10 00/14] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 01/14] net/idpf: add support for device start and stop Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 00/18] add support for idpf PMD in DPDK Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 01/18] common/idpf: introduce common library Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 02/18] net/idpf: add support for device initialization Junfeng Guo
2022-10-28 15:35 ` Andrew Rybchenko
2022-10-28 17:22 ` Xing, Beilei
2022-10-27 7:47 ` [PATCH v14 03/18] net/idpf: add Tx queue setup Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 04/18] net/idpf: add Rx " Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 05/18] net/idpf: add support for device start and stop Junfeng Guo
2022-10-28 15:45 ` Andrew Rybchenko
2022-10-27 7:47 ` [PATCH v14 06/18] net/idpf: add support for queue start Junfeng Guo
2022-10-28 15:50 ` Andrew Rybchenko
2022-10-28 17:34 ` Xing, Beilei
2022-10-27 7:47 ` [PATCH v14 07/18] net/idpf: add support for queue stop Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 08/18] net/idpf: add queue release Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 09/18] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 12/18] net/idpf: support parsing packet type Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 14/18] net/idpf: add support for RSS Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 15/18] net/idpf: add support for Rx offloading Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 16/18] net/idpf: add support for Tx offloading Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-27 7:47 ` [PATCH v14 18/18] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-29 3:27 ` [PATCH v15 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-29 3:27 ` [PATCH v15 01/18] common/idpf: introduce common library beilei.xing
2022-10-29 3:27 ` [PATCH v15 02/18] net/idpf: add support for device initialization beilei.xing
2022-10-29 3:27 ` [PATCH v15 03/18] net/idpf: add Tx queue setup beilei.xing
2022-10-29 3:27 ` [PATCH v15 04/18] net/idpf: add Rx " beilei.xing
2022-10-29 3:27 ` [PATCH v15 05/18] net/idpf: add support for device start and stop beilei.xing
2022-10-29 3:27 ` [PATCH v15 06/18] net/idpf: add support for queue start beilei.xing
2022-10-29 3:27 ` [PATCH v15 07/18] net/idpf: add support for queue stop beilei.xing
2022-10-29 3:27 ` [PATCH v15 08/18] net/idpf: add queue release beilei.xing
2022-10-29 3:27 ` [PATCH v15 09/18] net/idpf: add support for MTU configuration beilei.xing
2022-10-29 3:27 ` [PATCH v15 10/18] net/idpf: add support for basic Rx datapath beilei.xing
2022-10-29 3:27 ` [PATCH v15 11/18] net/idpf: add support for basic Tx datapath beilei.xing
2022-10-29 3:27 ` [PATCH v15 12/18] net/idpf: support parsing packet type beilei.xing
2022-10-29 3:27 ` [PATCH v15 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
2022-10-29 3:27 ` [PATCH v15 14/18] net/idpf: add support for RSS beilei.xing
2022-10-29 3:27 ` [PATCH v15 15/18] net/idpf: add support for Rx offloading beilei.xing
2022-10-29 3:27 ` [PATCH v15 16/18] net/idpf: add support for Tx offloading beilei.xing
2022-10-29 3:27 ` [PATCH v15 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
2022-10-29 3:27 ` [PATCH v15 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-29 14:48 ` [PATCH v15 00/18] add support for idpf PMD in DPDK Andrew Rybchenko
2022-10-31 2:26 ` Xing, Beilei
2022-10-31 3:36 ` [PATCH v16 " beilei.xing
2022-10-31 3:36 ` [PATCH v16 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 3:36 ` [PATCH v16 02/18] net/idpf: add support for device initialization beilei.xing
2022-10-31 3:36 ` [PATCH v16 03/18] net/idpf: add Tx queue setup beilei.xing
2022-10-31 3:36 ` [PATCH v16 04/18] net/idpf: add Rx " beilei.xing
2022-10-31 3:36 ` [PATCH v16 05/18] net/idpf: add support for device start and stop beilei.xing
2022-10-31 3:36 ` [PATCH v16 06/18] net/idpf: add support for queue start beilei.xing
2022-10-31 3:36 ` [PATCH v16 07/18] net/idpf: add support for queue stop beilei.xing
2022-10-31 3:36 ` [PATCH v16 08/18] net/idpf: add queue release beilei.xing
2022-10-31 3:36 ` [PATCH v16 09/18] net/idpf: add support for MTU configuration beilei.xing
2022-10-31 3:36 ` [PATCH v16 10/18] net/idpf: add support for basic Rx datapath beilei.xing
2022-10-31 3:36 ` [PATCH v16 11/18] net/idpf: add support for basic Tx datapath beilei.xing
2022-10-31 3:36 ` [PATCH v16 12/18] net/idpf: support parsing packet type beilei.xing
2022-10-31 3:36 ` [PATCH v16 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
2022-10-31 3:36 ` [PATCH v16 14/18] net/idpf: add support for RSS beilei.xing
2022-10-31 3:36 ` [PATCH v16 15/18] net/idpf: add support for Rx offloading beilei.xing
2022-10-31 3:36 ` [PATCH v16 16/18] net/idpf: add support for Tx offloading beilei.xing
2022-10-31 3:36 ` [PATCH v16 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
2022-10-31 3:36 ` [PATCH v16 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 5:15 ` [PATCH v17 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 5:15 ` [PATCH v17 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 5:15 ` [PATCH v17 02/18] net/idpf: add support for device initialization beilei.xing
2022-10-31 5:15 ` [PATCH v17 03/18] net/idpf: add Tx queue setup beilei.xing
2022-10-31 5:15 ` [PATCH v17 04/18] net/idpf: add Rx " beilei.xing
2022-10-31 5:15 ` [PATCH v17 05/18] net/idpf: add support for device start and stop beilei.xing
2022-10-31 5:15 ` [PATCH v17 06/18] net/idpf: add support for queue start beilei.xing
2022-10-31 5:15 ` [PATCH v17 07/18] net/idpf: add support for queue stop beilei.xing
2022-10-31 5:15 ` [PATCH v17 08/18] net/idpf: add queue release beilei.xing
2022-10-31 5:15 ` [PATCH v17 09/18] net/idpf: add support for MTU configuration beilei.xing
2022-10-31 5:15 ` [PATCH v17 10/18] net/idpf: add support for basic Rx datapath beilei.xing
2022-10-31 5:15 ` [PATCH v17 11/18] net/idpf: add support for basic Tx datapath beilei.xing
2022-10-31 5:15 ` [PATCH v17 12/18] net/idpf: support parsing packet type beilei.xing
2022-10-31 5:15 ` [PATCH v17 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
2022-10-31 5:15 ` [PATCH v17 14/18] net/idpf: add support for RSS beilei.xing
2022-10-31 5:15 ` [PATCH v17 15/18] net/idpf: add support for Rx offloading beilei.xing
2022-10-31 5:15 ` [PATCH v17 16/18] net/idpf: add support for Tx offloading beilei.xing
2022-10-31 5:15 ` [PATCH v17 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
2022-10-31 5:15 ` [PATCH v17 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 8:33 ` [PATCH v18 00/18] add support for idpf PMD in DPDK beilei.xing
2022-10-31 8:33 ` [PATCH v18 01/18] common/idpf: introduce common library beilei.xing
2022-10-31 8:33 ` [PATCH v18 02/18] net/idpf: add support for device initialization beilei.xing
2022-10-31 8:33 ` [PATCH v18 03/18] net/idpf: add Tx queue setup beilei.xing
2022-10-31 8:33 ` [PATCH v18 04/18] net/idpf: add Rx " beilei.xing
2022-10-31 8:33 ` [PATCH v18 05/18] net/idpf: add support for device start and stop beilei.xing
2022-10-31 8:33 ` [PATCH v18 06/18] net/idpf: add support for queue start beilei.xing
2022-10-31 8:33 ` [PATCH v18 07/18] net/idpf: add support for queue stop beilei.xing
2022-10-31 8:33 ` [PATCH v18 08/18] net/idpf: add queue release beilei.xing
2022-10-31 8:33 ` [PATCH v18 09/18] net/idpf: add support for MTU configuration beilei.xing
2022-10-31 8:33 ` [PATCH v18 10/18] net/idpf: add support for basic Rx datapath beilei.xing
2022-10-31 8:33 ` [PATCH v18 11/18] net/idpf: add support for basic Tx datapath beilei.xing
2022-10-31 8:33 ` [PATCH v18 12/18] net/idpf: support parsing packet type beilei.xing
2022-10-31 8:33 ` [PATCH v18 13/18] net/idpf: add support for write back based on ITR expire beilei.xing
2022-10-31 8:33 ` [PATCH v18 14/18] net/idpf: add support for RSS beilei.xing
2022-10-31 8:33 ` [PATCH v18 15/18] net/idpf: add support for Rx offloading beilei.xing
2022-10-31 8:33 ` [PATCH v18 16/18] net/idpf: add support for Tx offloading beilei.xing
2022-10-31 8:33 ` [PATCH v18 17/18] net/idpf: add AVX512 data path for single queue model beilei.xing
2022-10-31 8:33 ` [PATCH v18 18/18] net/idpf: add support for timestamp offload beilei.xing
2022-10-31 13:38 ` [PATCH v18 00/18] add support for idpf PMD in DPDK Thomas Monjalon
2022-10-27 5:44 ` [PATCH v13 02/18] net/idpf: add support for device initialization Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 03/18] net/idpf: add Tx queue setup Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 04/18] net/idpf: add Rx " Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 05/18] net/idpf: add support for device start and stop Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 06/18] net/idpf: add support for queue start Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 07/18] net/idpf: add support for queue stop Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 08/18] net/idpf: add queue release Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 09/18] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
2022-10-27 5:44 ` [PATCH v13 12/18] net/idpf: support parsing packet type Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 14/18] net/idpf: add support for RSS Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 15/18] net/idpf: add support for Rx offloading Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 16/18] net/idpf: add support for Tx offloading Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-27 5:45 ` [PATCH v13 18/18] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 02/18] net/idpf: add support for device initialization Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 03/18] net/idpf: add Tx queue setup Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 04/18] net/idpf: add Rx " Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 05/18] net/idpf: add support for device start and stop Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 06/18] net/idpf: add support for queue start Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 07/18] net/idpf: add support for queue stop Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 08/18] net/idpf: add queue release Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 09/18] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 10/18] net/idpf: add support for basic Rx datapath Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 11/18] net/idpf: add support for basic Tx datapath Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 12/18] net/idpf: support packet type get Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 14/18] net/idpf: add support for RSS Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 15/18] net/idpf: add support for Rx offloading Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 16/18] net/idpf: add support for Tx offloading Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-26 10:10 ` [PATCH v12 18/18] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 02/18] net/idpf: add support for device initialization Junfeng Guo
2022-10-25 8:57 ` Andrew Rybchenko
2022-10-26 8:28 ` Xing, Beilei
2022-10-28 15:14 ` Andrew Rybchenko
2022-10-28 17:19 ` Xing, Beilei
2022-10-24 13:12 ` [PATCH v11 03/18] net/idpf: add Tx queue setup Junfeng Guo
2022-10-25 9:40 ` Andrew Rybchenko
2022-10-26 8:34 ` Xing, Beilei
2022-10-24 13:12 ` [PATCH v11 04/18] net/idpf: add Rx " Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 05/18] net/idpf: add support for device start and stop Junfeng Guo
2022-10-25 9:49 ` Andrew Rybchenko
2022-10-26 8:38 ` Xing, Beilei
2022-10-24 13:12 ` [PATCH v11 06/18] net/idpf: add support for queue start Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 07/18] net/idpf: add support for queue stop Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 08/18] net/idpf: add queue release Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 09/18] net/idpf: add support for packet type get Junfeng Guo
2022-10-25 9:57 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 10/18] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 11/18] net/idpf: add support for basic Rx datapath Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 12/18] net/idpf: add support for basic Tx datapath Junfeng Guo
2022-10-25 10:12 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 13/18] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 14/18] net/idpf: add support for RSS Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 15/18] net/idpf: add support for Rx offloading Junfeng Guo
2022-10-25 10:03 ` Andrew Rybchenko
2022-10-28 1:48 ` Xing, Beilei
2022-10-24 13:12 ` [PATCH v11 16/18] net/idpf: add support for Tx offloading Junfeng Guo
2022-10-25 10:14 ` Andrew Rybchenko
2022-10-24 13:12 ` [PATCH v11 17/18] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-24 13:12 ` [PATCH v11 18/18] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 02/14] net/idpf: add support for queue start Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 03/14] net/idpf: add support for queue stop Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 04/14] net/idpf: add queue release Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 05/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 06/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 07/14] net/idpf: add support for basic Rx datapath Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 08/14] net/idpf: add support for basic Tx datapath Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 09/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 11/14] net/idpf: add support for Rx offloading Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 12/14] net/idpf: add support for Tx offloading Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-24 13:01 ` [PATCH v10 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-21 7:39 ` Andrew Rybchenko
2022-10-21 7:48 ` Andrew Rybchenko
2022-10-21 12:41 ` Zhang, Qi Z
2022-10-25 7:52 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-21 7:44 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-21 7:53 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-21 7:56 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-21 8:00 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-21 8:29 ` Andrew Rybchenko
2022-10-24 13:26 ` Xing, Beilei
2022-10-21 5:18 ` [PATCH v9 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-21 8:38 ` Andrew Rybchenko
2022-10-21 5:18 ` [PATCH v9 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-21 5:18 ` [PATCH v9 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-20 6:29 ` [PATCH v8 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 14:54 ` [PATCH v6 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 11:03 ` [PATCH v5 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 02/14] net/idpf: add support for device initialization Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 03/14] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 04/14] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 05/14] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 06/14] net/idpf: add support for device information get Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 07/14] net/idpf: add support for packet type get Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 08/14] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 09/14] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 10/14] net/idpf: add support for RSS Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 11/14] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 12/14] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 13/14] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-19 10:37 ` [PATCH v4 14/14] net/idpf: add support for timestamp offload Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 02/15] net/idpf: add support for device initialization Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 03/15] net/idpf: add queue setup and release in single queue model Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 04/15] net/idpf: add queue setup and release in split " Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 05/15] net/idpf: add support for queue start and stop Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 06/15] net/idpf: add support for device information get Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 07/15] net/idpf: add support for packet type get Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 08/15] net/idpf: add support for link status update Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 09/15] net/idpf: add support for basic Rx/Tx datapath Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 10/15] net/idpf: add support for Rx/Tx offloading Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 11/15] net/idpf: add support for RSS Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 12/15] net/idpf: add support for MTU configuration Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 13/15] net/idpf: add support for write back based on ITR expire Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 14/15] net/idpf: add AVX512 data path for single queue model Junfeng Guo
2022-10-18 11:12 ` [PATCH v3 15/15] net/idpf: add support for timestamp offload Junfeng Guo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).